Authenticate downloaded binaries

Summary:
This commit adds logic to authenticate all Bazel binaries that are
downloaded, as long as the user has GPG installed. If a user does not
have GPG installed, a new warning will be printed when a binary is
downloaded, but Bazelisk will function the same way as before. (GPG is
installed by default on Debian and Ubuntu.)

No new subprocesses are spawned when an already-downloaded version of
Bazel is run. The only appreciable overhead is incurred at download
time.

Resolves #15.

Test Plan:

  - Remove the `~/.bazelisk` directory. Run `./bazelisk.py version`.
    Note that it downloads the latest binary and the latest signature,
    then prints “Authenticity verified” before invoking Bazel.

  - Run `./bazelisk.py version` again. Note that it does not verify the
    signature.

  - Remove the `~/.bazelisk` directory. Symlink `/bin/false` to
    `~/bin/gpg`, and ensure that the symlink precedes the real `gpg` on
    your path. Run Bazelisk, and note that it prints a warning that GPG
    is not available but executes Bazel anyway. Run Bazelisk again, and
    note that it does not print the warning (because it reuses the
    existing executable without reauthenticating). Remove the symlink.

  - Remove the `~/.bazelisk` directory. Edit `bazelisk.py`, changing the
    `determine_urls` function so that the returned `binary_url` is an
    arbitrary web page (like `http://example.com/`) but the signature
    URL is unchanged. Run Bazelisk, and note that Bazelisk reports,
    “Failed to authenticate binary!”, includes the GPG output (“BAD
    signature”), and aborts with exit code 2 _without_ invoking Bazel.
    Run `ls ~/.bazelisk/bin` and note that it does not include the
    invalid binary (though the signature is still there). Revert the
    changes to `bazelisk.py`.

  - Remove the `~/.bazelisk` directory. Create an arbitrary document and
    use `gpg --detach-sign` to sign it with a key that is not the Bazel
    signing key. Spawn a web server (`python -m SimpleHTTPServer`) to
    serve the “malicious executable” and its signature. Edit
    `bazelisk.py`, changing the `determine_urls` function to point both
    the binary and the signature to this local web server. Run Bazelisk,
    and note that it fails to authenticate the binary, with the message
    “public key not found”.

Repeat the above steps in Python 2 and Python 3.

Verify that your personal GnuPG database has not been modified (in
particular, the Bazel key should not have been installed, and the trust
settings should not have been modified).

I have tested this on Linux with gpg (GnuPG) 1.4.20. I don’t see any
reason that it shouldn’t work on macOS or Windows as long as the gpg(1)
interfaces are the same.

wchargin-branch: authenticate-binaries

Author: wchargin

README.md

@@ -2,7 +2,7 @@
 
 **A user-friendly launcher for Bazel.**
 
-Bazelisk is a wrapper for Bazel. It automatically picks a good version of Bazel given your current working directory, downloads it from the official server (if required) and then transparently passes through all command-line arguments to the real Bazel binary. You can call it just like you would call Bazel.
+Bazelisk is a wrapper for Bazel. It automatically picks a good version of Bazel given your current working directory, downloads it from the official server (if required) and then transparently passes through all command-line arguments to the real Bazel binary. You can call it just like you would call Bazel. If you have [`gpg`][GnuPG] installed, Bazelisk will authenticate all Bazel downloads.
 
 Bazelisk is currently not an official part of Bazel and is not tested or code reviewed as thoroughly as Bazel itself. It's a personal project that @philwo (a core contributor to Bazel) wrote in his free time. If users like it, we might merge it into the bazelbuild organization and make it an official tool.
 
@@ -29,6 +29,10 @@ In the future I will add support for release candidates and for building Bazel f
 
 For ease of use, Bazelisk is written to work with Python 2.7 and 3.x and only uses modules provided by the standard library.
 
+If [GnuPG] is installed and `gpg` is available on the system path, Bazelisk will verify the integrity of the binaries that it downloads.
+
+[GnuPG]: https://www.gnupg.org/
+
 ## Ideas for the future
 
 - Add a Homebrew recipe for Bazelisk to make it easy to install on macOS.

bazelisk.py

@@ -15,6 +15,7 @@
 limitations under the License.
 """
 
+import collections
 from contextlib import closing
 from distutils.version import LooseVersion
 import json
@@ -24,6 +25,7 @@
 import shutil
 import subprocess
 import sys
+import tempfile
 import time
 
 try:
@@ -34,6 +36,10 @@
 
 ONE_HOUR = 1 * 60 * 60
 
+# Bazelisk exits with this code when GPG is installed but the binary
+# cannot be authenticated.
+AUTHENTICATION_FAILURE_EXIT_CODE = 2
+
 
 def decide_which_bazel_version_to_use():
   # Check in this order:
@@ -116,25 +122,152 @@ def normalized_machine_arch_name():
   return machine
 
 
-def determine_url(version, bazel_filename):
+SubprocessResult = collections.namedtuple("SubprocessResult", ("exit_code",))
+
+
+def subprocess_run(command, input=None, error_message=None):
+  """Kind of like Python 3's subprocess.run, but works in Python 2.
+
+  The contents of stdout and stderr are captured. If the command
+  succeeds (exit code 0), they are not printed. If the command fails,
+  stderr is printed along with the provided error message (if any).
+
+  Args:
+    command: The command to be executed, as a list of strings
+    input: A bytestring to use as stdin, or None.
+    error_message: If not None, will be logged on failure.
+
+  Returns:
+    A `SubprocessResult` including the process's exit code.
+  """
+  process = subprocess.Popen(
+      command,
+      stdin=subprocess.PIPE,
+      stdout=subprocess.PIPE,
+      stderr=subprocess.PIPE)
+  (stdout, stderr) = process.communicate(input=input)
+  exit_code = process.wait()
+  if exit_code != 0 and error_message is not None:
+    if error_message is not None:
+      sys.stderr.write("bazelisk: {}\n".format(error_message))
+    write_binary_to_stderr(stderr)
+  return SubprocessResult(exit_code=exit_code)
+
+
+def write_binary_to_stderr(bytestring):
+  # Python 2 compatibility hack. In Python 3, you can't write byte
+  # strings to stdio; instead, you have to use the `sys.stderr.buffer`
+  # attribute, which is not available in Python 2.
+  buffer = getattr(sys.stderr, "buffer", sys.stderr)
+  buffer.write(bytestring)
+
+
+def verify_authenticity(binary_path, signature_path):
+  """Authenticate a binary and signature against the Bazel public key.
+
+  This will use a fresh temporary keyring populated only with the
+  Bazel team's signing key; it is independent of any existing PGP data
+  or settings that the user may have.
+
+  Args:
+    binary_path: File path to the Bazel binary to be executed.
+    signature_path: File path to the detached signature made by the
+      Bazel release PGP key to sign the provided binary.
+
+  Returns:
+    True if the binary is valid or gpg is not installed; False if gpg is
+    installed but we cannot determine that the binary is valid.
+  """
+  if subprocess_run(
+      ["gpg", "--batch", "--version"],
+      error_message=
+      "Warning: skipping authenticity check because GPG is not installed.",
+  ).exit_code != 0:
+    return True
+
+  tempdir = tempfile.mkdtemp(prefix="tmp_bazelisk_gpg_")
+  try:
+    gpg_invocation = [
+        "gpg",
+        "--batch",
+        "--no-default-keyring",
+        "--homedir",
+        tempdir,
+    ]
+    if subprocess_run(
+        gpg_invocation + ["--import-ownertrust"],
+        input=BAZEL_ULTIMATE_OWNERTRUST,
+        error_message="Failed to initialize GPG keyring").exit_code != 0:
+      return False
+    if subprocess_run(
+        gpg_invocation + ["--import"],
+        input=BAZEL_PUBLIC_KEY,
+        error_message="Failed to import Bazel public key").exit_code != 0:
+      return False
+    if subprocess_run(
+        gpg_invocation + ["--verify", signature_path, binary_path],
+        error_message="Failed to authenticate binary!").exit_code != 0:
+      return False
+    sys.stderr.write("Verified authenticity.\n")
+    return True
+
+  finally:
+    shutil.rmtree(tempdir)
+
+
+DownloadUrls = collections.namedtuple("DownloadUrls",
+                                      ("binary_url", "signature_url"))
+
+
+def determine_urls(version, bazel_filename):
   # Split version into base version and optional additional identifier.
   # Example: '0.19.1' -> ('0.19.1', None), '0.20.0rc1' -> ('0.20.0', 'rc1')
   (version, rc) = re.match(r'(\d*\.\d*(?:\.\d*)?)(rc\d)?', version).groups()
-  return "https://releases.bazel.build/{}/{}/{}".format(
+  binary_url = "https://releases.bazel.build/{}/{}/{}".format(
       version, rc if rc else "release", bazel_filename)
+  signature_url = "{}.sig".format(binary_url)
+  return DownloadUrls(binary_url=binary_url, signature_url=signature_url)
+
+
+def download_file(url, destination_path):
+  """Download a file from the given URL, saving it to the given path."""
+  sys.stderr.write("Downloading {}...\n".format(url))
+  with closing(urlopen(url)) as response:
+    with open(destination_path, 'wb') as out_file:
+      shutil.copyfileobj(response, out_file)
 
 
 def download_bazel_into_directory(version, directory):
+  """Download and authenticate the specified version of Bazel.
+
+  If the binary already exists, it will not be re-downloaded.
+
+  If the binary does not exist, it and its signature will be downloaded.
+  The binary will only be saved and made executable if the signature is
+  valid (or if we are unable to validate the signature because GPG is
+  not installed).
+
+  If the signature is invalid, a `SystemExit` exception will be raised.
+
+  Returns:
+    The path to the valid, executable Bazel binary within the provided
+    directory.
+  """
   bazel_filename = determine_bazel_filename(version)
-  url = determine_url(version, bazel_filename)
-  destination_path = os.path.join(directory, bazel_filename)
-  if not os.path.exists(destination_path):
-    sys.stderr.write("Downloading {}...\n".format(url))
-    with closing(urlopen(url)) as response:
-      with open(destination_path, 'wb') as out_file:
-        shutil.copyfileobj(response, out_file)
-  os.chmod(destination_path, 0o755)
-  return destination_path
+  urls = determine_urls(version, bazel_filename)
+  binary_path = os.path.join(directory, bazel_filename)
+  if not os.path.exists(binary_path):
+    untrusted_binary_path = "{}.untrusted".format(binary_path)
+    signature_path = "{}.sig".format(binary_path)
+    download_file(urls.binary_url, untrusted_binary_path)
+    download_file(urls.signature_url, signature_path)
+    if verify_authenticity(untrusted_binary_path, signature_path):
+      os.rename(untrusted_binary_path, binary_path)
+    else:
+      os.unlink(untrusted_binary_path)
+      raise SystemExit(AUTHENTICATION_FAILURE_EXIT_CODE)
+  os.chmod(binary_path, 0o755)
+  return binary_path
 
 
 def maybe_makedirs(path):