PDF, Portable Document Format

Ghostscript

🏗️

QPDF

• QPDF: A Content-Preserving PDF Transformation System
• qpdf/qpdf: Primary QPDF source code and documentation
• QPDF Manual

QPDF is a command-line program that does structural, content-preserving transformations on PDF files. It could have been called something like pdf-to-pdf. It also provides many useful capabilities to developers of PDF-producing software or for people who just want to look at the innards of a PDF file to learn more about how they work. … QPDF includes support for merging and splitting PDFs through the ability to copy objects from one PDF file into another and to manipulate the list of pages in a PDF file. Th…

QPDF is not a PDF content creation library, a PDF viewer, or a program capable of converting PDF into other formats. In particular, QPDF knows nothing about the semantics of PDF content streams. If you are looking for something that can do that, you should look elsewhere. However, once you have a valid PDF file, QPDF can be used to transform that file in ways perhaps your original PDF creation can’t handle. For example, programs generate simple PDF files but can’t password-protect them, web-optimize them, or perform other transformations of that type.

Pretend I printed then scanned the PDF

Many business demand “wet ink” signatures on signed PDFs, i.e. that I fill the digital form in, then print it, sign it and then scan it in again. This wastes time, money, paper and adds no discernable security to the truly shitty security mechanisms that is signing things with ink. Of course, one must always do this honestly when it is a substantial contract.

Now I am not lawyer, but I expect that the following method might be much easier, cheaper, no less secure and for all purposes indistinguishable from printing and scanning a wet ink signature.

We cannot fix the lack of security, but we can automate fake-printing so that it at least saves time and money. Here are many scripts to simulate a round trip via the printer:

• Show HN: Make Your PDF Look Scanned | Hacker News
• Make a pdf look scanned using ImageMagick

I quite like

convert -density 100 input.pdf -rotate "$([ $((RANDOM % 2)) -eq 1 ] && echo -)0.$(($RANDOM % 4 + 5))" -attenuate 0.4 +noise Multiplicative -attenuate 0.03 +noise Multiplicative -sharpen 0x1.0 -colorspace Gray output.pdf

The scripts get much more elaborate than that though.

Searching

Shawn graham suggests a pdftotext hack. First, install poppler using your choice of package manager. Now,

find /MLR -name '*.pdf' -exec sh \
  -c 'pdftotext "{}" - | grep --with-filename --label="{}" \
  --color "SEARCHTERM"' \;

Reduce bloat

PDFs have a lot of ways of storing data and many of them leave you keeping lots of crap there that you do not need for the current purpose, in the form of, presumably, inefficiently compressed images, excessively high resolution images, or miscellaneous other crap. Slimming them down to the essentials is in general complicated and context-dependent, and I do not know of a general solution. Here are some that work in some contexts.

Classic ghostscript method:

gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.5 \
  -dPDFSETTINGS=/ebook \
  -dNOPAUSE -dQUIET -dBATCH \
  -sOutputFile=output.pdf input.pdf

Wrapped up into a nice little script, ShrinkPDF: (90 is the dpi here.)

./shrinkpdf.sh in.pdf out.pdf 90

There is also cpdf and the GUI version Densify.

Commercial online PDF tool smallpdf claims to shrink PDFs also. (USD12/month with free trial.)

A peer recommendation: PDF Resizer.

Concatenate/split

I need to do this so often that I may yet get around to making a keybaord shortcut for it.

This is where qpdf excels, although ghostscript is older and so has more HOWTOs.

This ghostcript command concatenates PDFs:

gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite \
  -dPDFSETTINGS=/prepress -sOutputFile=output.pdf input*.pdf

The QPDF version is not much more intuitive:

qpdf --empty --pages *.pdf -- out.pdf

You can split PDFs also with ghostscript, but usually you want a GUI to see what you are splitting, no?

pdfunite is the poppler command that does this. Popper is fairly ubiquitous also

PDFMix and PDF shuffler have both been recommended to me for this.

Antoine Chambert-Loir points out the underdocumented command texexec has many PDF-editing sub-commands.

The following command extract pages 1 to 5, page 7 and pages 8 to 12 from file.pdf and puts it in outputfile.pdf

texexec --pdfselect --select=1:5,7,8:12 --result=outputfile.pdf file.pdf

There is also PDFtk which has a handy command line. This has been ported to linux as pdftk-java and also has a gui called PDF Chain.

PDF to text

OCRMyPDF makes a scanned PDF possibly-searchable and also optimizes the size, optionally aggressively. This will not downsample, but it will get better monochrome compression than normal.

Dangerzone is more extreme still; it deliberately rasterizes the PDF and deletes all metadata to make it anonymous and safe before OCRing it to get the text out:

Dangerzone works like this: You give it a document that you don’t know if you can trust (for example, an email attachment). Inside of a sandbox, dangerzone converts the document to a PDF (if it isn’t already one), and then converts the PDF into raw pixel data: a huge list of RGB color values for each page. Then, in a separate sandbox, dangerzone takes this pixel data and converts it back into a PDF.

Read the blog post for more information.

EPS to PDF

EPS to PDF conversion:

ps2pdf14 -dEPSCrop Logo.eps

PDF to SVG

pdf2svg generates editable vector diagrams from the PDF.

HTML to PDF

See weasyprint, below.

To CMYK

CMYK Color conversion of RGB PDF with GhostScript:

gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite \
  -sColorConversionStrategy=CMYK \
  -sColorConversionStrategyForImages=CMYK \
  -sDEVICE=pdfwrite \
  -dProcessColorModel=/DeviceCMYK \
  -dCompatibilityLevel=1.5 \
  -sOutputFile=result_cmyk.pdf \
  your_document.pdf

See also a PDF to TIFF example.

To Greyscale

gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite \
  -sColorConversionStrategy=Gray \
  -dProcessColorModel=/DeviceGray \
  -dCompatibilityLevel=1.5 \
  -sOutputFile=result_gray.pdf \
  your_document.pdf

Weasyprint

Weasyprint seems the cleanest. It converts HTML+CSS into PDF, and is written in pure python. It can be used from the command line or programatically.

It is based on various libraries but not on a full rendering engine like Blink, Gecko or WebKit. The CSS layout engine is written in Python, designed for pagination, and meant to be easy to hack on.

pip install weasyprint
weasyprint https://weasyprint.org/ weasyprint.pdf

SVGlib

svglib provides a pure python library that can convert SVG to PDF, and a command line utility for same, svg2pdf. Thus one can, e.g. add SVGs to PDFs in reportlab.

Reportlab

Apropos that, reportlab is the famed monstrous classic way of programatically generating PDFs from code. It includes a modicum of typesetting. It doesn’t edit PDFs so much, but it generates them pretty well. Its integration with other things is often weak — if you thought that inserting LaTeX equations would be simple, or HTML snippets etc. On the other hand it has fancy features such as its own chart generation library. On the third hand, there are better, more widely supported charting libraries that it doesn’t use. Litmus test: Use it if the following feels to you like a natural way to print two columns:

from reportlab.platypus import (
  BaseDocTemplate,
  Frame,
  Paragraph,
  PageBreak,
  PageTemplate )
from reportlab.lib.styles import getSampleStyleSheet
import random

words = (
  "lorem ipsum dolor sit amet consetetur "
  "sadipscing elitr sed diam nonumy eirmod "
  "tempor invidunt ut labore et").split()

styles=getSampleStyleSheet()
Elements=[]

doc = BaseDocTemplate(
  'basedoc.pdf',
  showBoundary=1)

#Two Columns

frame1 = Frame(
  doc.leftMargin,
  doc.bottomMargin,
  doc.width/2-6,
  doc.height,
  id='col1')
frame2 = Frame(
  doc.leftMargin+doc.width/2+6,
  doc.bottomMargin,
  doc.width/2-6,
  doc.height,
  id='col2')

Elements.append(
  Paragraph(
    " ".join([random.choice(words) for i in range(1000)]),
    styles['Normal']))
doc.addPageTemplates([
  PageTemplate(id='TwoCol',frames=[frame1,frame2]),
])

#start the construction of the pdf
doc.build(Elements)

pdfrw

pdfrw:

pdfrw is a Python library and utility that reads and writes PDF files:

• Operations include subsetting, merging, rotating, modifying metadata, etc. […]
• Can be used either standalone, or in conjunction with reportlab to reuse existing PDFs in new ones

Here is a gentle HOWTO. You can use it to put matplotlib plots in reportlab PDFs, getting the best of two bad worlds.

Scribus

scribus is a reasonable open-source desktop publishing tool. If your content is not amenable to automatic layout out it is a good choice, for e.g. posters. It includes a Python API, albeit a reputedly quirky one, which is AFAICT Python 2. For all that, it’s a simple and interactive way of generating PDFs programmatically, so might be worth it.