Tools and tips round up: CrowdTangle comes to an end and inside Archive.org's Collections
Plus: new tools for Twitter, Facebook, domain lookups and audio verification.
It feels like I keep having to start my tools and tips posts with updates about the disappearance of tools and resources. That’s life in the world of digital investigations, where products, tools, features and exploits come and go. Platforms giveth and they taketh away.
Here’s the latest bad news: As noted by Sector035 and others, Snapchat removed the freely accessible web version of the Snap Map. It’s still available in the app but obviously that’s less ideal.
We also got confirmation of something that people in the journalism and research community have been expecting and dreading: CrowdTangle will be shut off in August. CrowdTangle is a great tool for monitoring and searching Facebook and Instagram. This represents the end of an era for me, as I was one of the first journalists to use it as a tool for investigations. But I cut back my use of CrowdTangle when it became clear that Meta planned to kill it.
If you want more context and background, I recommend this post by CrowdTangle founder Brandon Silverman. (No relation, and he’s no longer with Meta.) He calls the decision to turn the tool off right before the US election’s critical final months “incredibly irresponsible.”
There’s also an open letter from research institutions, advocacy organizations, academics and others calling on Meta to keep CrowdTangle active until January 2025.
They write:
Meta’s decision will effectively prohibit the outside world, including election integrity experts, from seeing what’s happening on Facebook and Instagram — during the biggest election year on record. This means almost all outside efforts to identify and prevent political disinformation, incitements to violence, and online harassment of women and minorities will be silenced. It’s a direct threat to our ability to safeguard the integrity of elections.
Internet Archive Collections
One a more positive front, I recently became aware of the Internet Archive’s Collections feature. It let’s you perform keyword searches within specific archive collections, including dead news sites such as Gawker as well as in collections dedicated to .gov resources, Telegram, Pastebin and more.
I learned about it thanks to a helpful tip Ben Welsh shared on Twitter.
Mark Graham, the director of the Wayback Machine, told me that the Internet Archive launched Collections roughly two years ago. “We have been adding a new collection or two every few weeks,” he said. I asked him why and how they select content worthy of a Collection:
Various criteria, including the relative importance, significance or value of the collection to others, with a bias toward news and certain material that may no longer be available via the web. E.g. Gawker, FOIAonline.gov, Hong Kong news orgs that have been shut down (factwire.com, hk.appledaily.com, hkcnews.com and thestandnews.com) and others. We have tired to index Collections that might be useful to people.
He said the organization welcomes suggestions for new Collections to index. And don’t forget to read my two part series about using the Wayback Machine:
Finally, I recently published a follow up story to our Walmart fraud investigation. In the new story, we look at what happened when Walmart acquired a banking app called One and implemented changes including weakening fraud protections.
Remember you can always reach out with story tips or feedback!
Tools and tips
📍 Social Listening Specialist is a custom GPT that can analyze data gathered from social platforms. It also says it can analyze images. I asked it whether it can collect data directly from platforms and it said, “While I don't directly access these platforms in real-time due to my current setup, I can provide insights and analysis based on data you provide or through web browsing capabilities to understand trends and case studies.”
📍 Henk van Ess launched Twitter List Search, a “tool to search for Twitter lists using keyword(s).
📍 He also launched Facebook Hashtag Search, a “tool to search for Facebook hashtags using a keyword.” His Substack is here.
📍 The folks at myosint.training noted that OSINT Combine updated its really useful OSINT bookmarks file. This is a well organized resource of tools.
📍 Tweet Machine is a“bash script which retrieves links to the deleted (and other) tweets and replies of any Twitter user from WayBackMachine.” via
📍 Analystresearchtools.com has a nice interface for searching phone numbers, emails, license plates and other information. It also has a bulk URL opener and list of OSINT resources. via
📍 The amazing
also maintains a great collection of cheat sheets on GitHub. Topics include reverse face search, image OSINT, Telegram OSINT, and more!📍 Botomoter is one of many tools that died as a result of Twitter/X’s new restrictive API limits. But its creators have kept it online in archive mode. More on that: “Unlike the original Botometer that fetched data from Twitter and calculated bot scores on the fly, Botometer X is in archival mode and relies on pre-calculated scores based on data collected before June 2023.”
📍 Sector035 dedicated an edition of his excellent This Week in OSINT newsletter to sharing useful links and information about using search engines other than Google.
📍 Domain Digger “is an easy-to-use domain lookup tool that shows you DNS, WHOIS, Certificates, and subdomains in a single screen.” via
📍 Sherlock Eye is “an email search tool that’s similar to haveibeenpwned with a few enhancements. The UI is very intuitive and well-designed and is a project worth watching.” via
📍 Awesome Privacy is a “guide to finding and comparing privacy-respecting alternatives to popular software and services.”
📍 Prisa Media, a Spanish media company, launched VerificAudio, a tool to help identify deepfake audio in Spanish. More on audio fake detection below. Also, the Global Investigative Journalism Network is holding free webinar on April 9, “Webinar: Investigating Elections: Threat from AI Audio Deepfakes.”
📍 Ginger T shared a tip on how to create a Twitter/X bookmark for a hashtag search now that TweetDeck is no longer free.
📍 Kirby Plessas shared a method for finding the ID of a Facebook page in light of recent changes by Meta.
Worth reading
📚 Zach Seward, editorial director of AI initiatives at The New York Times, gave a really interesting talk with tons of useful examples at SXSW, “AI news that’s fit to print”
📚
wrote “A 1-minute way to geolocate road signs that show the distance to the nearest cities.”📚 She also wrote, “5 simple tricks to quickly analyze a larger list of URLs”
📚 NBC News published a fun quiz to test how good you are at identifying real versus AI-created audio. I also encourage you to read this comprehensive piece from GIJN, “How to Identify and Investigate AI Audio Deepfakes, a Major 2024 Election Threat”
📚 Jon Keegan of The Markup wrote, “I Used ChatGPT as a Reporting Assistant. It Didn’t Go Well”
📚 Reporter Jessica Mathews wrote, “A business reporter’s guide to documents.” It includes of great links to American company resources like the SEC, NLRB, and more.
📚 Adalytics published, “Are Amazon and other ad tech vendors serving ANA members’ ads on “Made for Arbitrage” websites in 2024?”
📚 Thomas Caliendo wrote, “How death records OSINT can help in genealogy research”
📚 GIJN published, “Unlocking the Secrets of Algorithms: Q & A with Investigative Data Journalist Leon Yin”
📚 Margot Williams wrote, “Research in Archives 101: How to find primary sources in archival collections”
📚 Christina Lekati wrote, “OSINT Techniques for Sensitive Documents That Have Escaped Into The Clear Web”
📚 Gabòr Friesen wrote, “Making Geolocation Proofs, 2023”
📚 Cecilie S. Traberg, Trisha Harjani, Jon Roozenbeek & Sander van der Linden wrote “The persuasive effects of social cues and source effects on misinformation susceptibility”
📚 Mark Coddington and Seth Lewis wrote, “Avoiding the news isn't the same as not consuming it”
Worth watching
Authentic8 posted six free videos of presentations from its recent OSINTUp event. I enjoyed the “How to analyze a video” session with @josemonkey, but they’re all interesting!
Bellingcat launched a video series to show you how to use more technically complex tools. Here’s a video on how to use the command line.
That’s it for this edition of Digital Investigations! Thanks for reading. You can find me on Threads, Bluesky, Mastodon, and LinkedIn. I’m not very active on Twitter these days.