Launching Blacklight Query, a Tool to Scan Websites for Privacy at Scale  – The Markup

Blacklight

Inspect batches of websites for user-tracking tech with our new open-source command-line tool By Miles Hilton

Imagine you’re a journalist investigating the privacy practices of hundreds of government websites. You hope to use Blacklight, The Markup’s real-time digital privacy inspector. But when you visit the tool on The Markup’s website, you realize there’s a catch: you’ll have to enter each website individually, then manually download the inspection results. Hours later and a hundred URLs in, you might be wondering, “Can I do this faster?”

We hear you. We want it to be as easy as possible to use our tools. That’s why we’re releasing Blacklight Query, an open-source command-line tool that makes it easy to quickly inspect a batch of websites with Blacklight.

With Blacklight Query, you can scan whole lists of URLs from the comfort of your very own terminal, without writing any code. Simply enter your target absolute urls into a text file and run a command to generate a folder of Blacklight results.

This video requires JavaScript to play.

Blacklight Query is built on Blacklight Collector, the open-source engine driving Blacklight. For those familiar with Blacklight, it’s important to note that while the two tools collect the same scan data, they differ in how they report results; for example, Blacklight Query does not filter third-party cookies to determine which are advertising-related trackers.

For more details on the capabilities of Blacklight Query and how it differs from The Markup’s site version of Blacklight, see its README, Blacklight Collector’s README, or reach out to us at [email protected].

User feedback is how we developed this tool, and helps us decide which direction to take Blacklight. Reach out to us at [email protected] with suggestions, questions, or just to share how you use Blacklight.

At The Markup, we believe in transparency and that everyone deserves access to our journalism and tools. We publish the data behind our investigations by default, do not track users, and have released two privacy-forward WordPress plugins and our interactive graphics template. Giving users ownership and insight into our tools by making our software open-source is a way for us to walk this talk. Blacklight Query is the next step on this path—a way to share what we’ve developed with our community and bring more transparency to the world of online privacy.

So draw up your list of sites, press “play,” and grab yourself a cup of coffee. The scans will be waiting for you when you return.

Your donations power our award-winning reporting and our tools. Together we can do more. Give now.

Launching Blacklight Query, a Tool to Scan Websites for Privacy at Scale

From the series — Blacklight

Miles Hilton Full Stack Operations Engineer