Public data is a gold mine of information. We are the miners.
Imagine you want to pitch a journalist on a story about your business. Is there a fast way to know which one has the right domain expertise, would write the best story, and get you the broadest reach?
The information that goes into that decision is publicly available, but the time it would take to collect and analyze the relevant information is daunting.
But this use case is a prime example for how technology can accelerate the creation of OSINT, or Open-Source Intelligence.
Open-Source Intelligence (OSINT) is a military and data science-y term for insights gleaned from public data. In this article, we take you back to the basics. We will explain the history of OSINT, how to transform information into intelligence, and our role in bringing OSINT to previously unexposed industries.
What is Open-Source Intelligence (OSINT)?
OSINT is valuable insights (aka intelligence) gleaned from publicly available information. Those insights are gathered and used for a specific purpose (eg, growing a business).
While the concept of OSINT is as old as time, the specific term “OSINT” has US military origins dating back to the 1940s and the monitoring of foreign broadcast transmissions. In these early cases, the military relied on manual information gathering: humans listened to and read as much as possible.
Within the past 20 years, OSINT has gained in popularity outside of military parlance. In particular, OSINT–as a term and a concept–has increasingly become a focus of business intelligence functions. One driving factor is the exponential growth in the types and volume of public data, including social media, financial records, satellite data, academic publications, institutional data, even things like box office sales and sports results! 
Google search trend for the term “OSINT” from 2004-present
Transforming Data from “Information” to “Intelligence”
With more data than ever before from which to derive intelligence, it is impossible for humans to collect and analyze it all. Gone are the days of dozens of people listening to foreign broadcasts. Today, businesses are turning to technology solutions for help. Here is a 3-step approach for turning information into intelligence.
Step 1: What Data is Relevant?
Of the billions of public data sources, we must first determine which data is relevant. Here are the questions that we work with business leaders to answer. These answers serve as the foundation of our data collection process:
- Which public data is relevant to my organization?
- Which public data, when overlaid with my proprietary data set, will produce valuable insights?
- Where might I find that public data?
- What formats and languages is that public data available in? Which formats and languages are relevant to my organization?
Step 2: Ranking, Extracting, and Standardizing the Information
Once the relevant information has been gathered, our technology reads and prioritizes the results. For example, it is not helpful to have 10,000 tweets about a relevant topic since nobody will take the time to read them. We must determine, of those 10,000 tweets, which are the top 20, 50, or 100 that are most important.
Once we know the most important information, we must extract and standardize it. This is particularly important when working with data sources across languages and formats.
Step 3: Analyzing the Information
Once the data has been collected, analysts and researchers undertake the task of deriving insights and intelligence that can aid in predicting patterns and behaviors and guiding critical decision-making processes.
And so information gets transformed into intelligence.
Some Examples of OSINT in Action
We love seeing how businesses use public data to make impactful changes. Here are just a few examples of how OSINT makes a difference:
- Our team at Arboretica developed a tracker that shows what legislation and investment capital is targeting climate solutions.
- The University of Helsinki’s Wildlife Trade project identified illegal wildlife trade on social media using machine learning.
- Bellingcat, an international collective of journalists, used open source satellite and trade data to uncover an illegal shipment of sarin, a component frequently used in chemical weapons.
- Algorithms can identify social media “bots” and “fake news” sites based on patterns in those entities’ behavior.
Arboretica’s Role in OSINT
There’s an old saying: “Many hands make light work.” When it comes to OSINT, we at Arboretica are the many hands.
When business leaders want to find insights in public data, they turn to us. Our proprietary technology aggregates, evaluates, and prioritizes the most important public data for each of our clients. And we are driven by our commitment to make the previously laborious tasks of data collection and analysis as efficient and scalable as possible, all in the name of advancing our mission of bringing value from public information to the masses.
If you are a business leader interested in exploring how public data can scale your business, contact us.
