OSINT Tools – Pt.3

Intro

Now that we’ve laid some theoretical foundation as to what OSINT consists of, let’s check out some tools and see how they can benefit us, as well as what are some of the most common uses. Before going any further, we would just like to quickly go over what types of information gathering there are, as well as some distinctions when it comes to these tools.

Active vs. Passive Recon

Within the context of an investigation, be it a penetration test or due diligence, we will use OSINT to gather some information.

The main distinction to be made here is active versus passive reconnaissance. Active reconnaissance means we are making some sort of a contact with a system we’re investigating. We interact with said systems. Some can be almost harmless, like ping, but some are much more intrusive, and can even mean brute forcing, and other such probing – which might be seen as hacking regardless of the fact that the resources are indeed in the open.

In general, in such a way we also might leave traces in the form of logs – which can further show the length of the connection, our IP address, etc.

When we are doing passive reconnaissance, we are not interacting with the systems. We might look up our target on Shodan, which would be considered passive, since we’re just using data that’s already out there, and are in no way interacting with any of the systems of interest.

There are merits to both sides, however, it’s crucial that we are aware of the distinction, so as to not hinder our investigation – we need to know what to use, and when.

Types of OSINT Tools

Based on what the tool does, we can say there are three main categories:

• Aggregation Tools
• Discovery Tools
• Scraping Tools

Discovery Tools – tools that enable us to query and search the data that is already out there. The best example is Google search engine. Seemingly simple, but Google has a lot of websites indexed and crawled, which in turn gives us enormous potential when it comes to discovering new information. Another example would be Shodan.

Aggregation Tools – these tools help us connect the dots, so to speak, once we have gathered all of our relevant data and are in need of further relating it, and compiling it into a functional, easily digestible, format.

Scraping Tools – when we have successfully discovered the information we need, we would like to extract it in an easy and safe way. With these tools, we can avoid extracting anything that is of no use to us, as well as saving our precious resources e.g. time and bandwidth.

With all of that being said, there are a plethora of tools out there, but we have decided to give a brief overview of a few that we felt are the most essential ones. It’s up to you to establish your own methodology, and do research accordingly, as there is no exact path one would follow when conducting OSINT investigations.

Google Search Engine – Google Dorking

Beside your everyday uses of Google’s search engine, there’s a lot of options for you to refine your queries.

A simplest example is adding quotation marks to your search. By doing so, Google will interpret whatever we’ve put inside the quotation marks as an exact phrase, and will give us only the results where that exact phrase comes up.

Another common example is adding the term site to our search. If we wanted to search for let’s say imdb new movies we would get something like this… notice the number of results.

On the other hand, if we were to add site: to our search, we would get a result similar to this…

As we can see, there’s a drastic difference in the number of results obtained, just by leveraging one of the many Google dorks.

We can even look for specific filetypes, with the filetype keyword.

If we want to look for publicly available .pdf’s for example, we can add the keyword like this:

We can also say intitle – and Google will return results if the exact phrase appears in the title of the page; there’s cache too – which will give us Google’s cached version of the URL that we’ve specified.

There are many more dorks available, and this is a big topic which we will look to cover in an article dedicated just to Google dorking.

But for now, we’d like to mention that this is completely legal as we are querying against legal, publicly available information. Of course, be mindful that what you do with the information might not be legal.

Shodan.io

With Internet connected devices number being higher than ever, a search engine dedicated to IoT – Internet of Things – Shodan is an irreplaceable tool to have in your arsenal.

If, for example, publicly accessible CCTV cameras are something that you might be looking into, Shodan’s got you covered.

Heck, if you want to check if your smart fridge is publicly accessible, Shodan can help you!

To use Shodan fully, you’ll need a paid subscription, however you might start with the free tier – but you’ll only get a limited amount of searches.

Best free(mium) alternative to Shodan is Censys which also tries to discover, analyze, and monitor Internet accessible devices.

OSINT Framework

The OSINT Framework is one of the most popular OSINT tools out there, and rightly so. Structured like a web directory of tools, it has almost everything you might need for your investigation, which makes it an extremely attractive option for information gathering.

Also, most of the tools in this web directory are directly usable and accessible through a browser, which is a great thing to have, since almost all of the best OSINT tools are created for Linux. Thus, the OSINT Framework provides us with a very useful and accessible bundle of tools, regardless of the platform – which is extremely valuable.

It is worth noting that most of the tools found within are free, with only a minority being premium, subscription based tools.

Maltego

Maltego is a wonderful aggregator of interfaces to various OSINT databases – from the official Maltego website – https://www.maltego.com/.

With Maltego, we can investigate and find information on organizations, individuals, as well as investigate cryptocurrencies, and much, much, more.

Once registered (which can be done for free – as a part of community license) you are brought to a GUI from which you can start your investigation. Results of your queries (Maltego calls them transforms) are displayed in a beautiful bubble graph, which maps the relations between your nodes.

 

Maltego starting screen

In our example search, where we’ve chosen Domain as Maltego entity, for youtube.com, we’ve obtained the following:

As we can see from the image, on the bottom are the transforms that were run, and on our graph we see color-coded results of our query. We’ve got 148 entities, and some of those include MX and NS records, email addresses, people, phone numbers, emails, etc.

We just ran the all transforms search, of course, in reality we would maybe use only transforms that we need, or we would install specific modules (from Maltego starting page), so that we can query for information that’s relevant for our investigation. Some of the modules that we can install are paid, but there are also some good free ones.

Maltego definitely warrants an article of its own, but we wanted to briefly show what this awesome tool is all about. Oh, and one more thing – Maltego runs on Linux, Windows, and MacOS.

Recon-ng

Another great tool is Recon-ng. This is a completely free, open source, CLI tool made for web-based open source reconnaissance.

It is completely modular, it has its own default modules that are also open source, while also having a marketplace from which we can further enrich it with whatever we might need.

The information we collect with it is stored in a database, which means we can use it to generate custom reports, if that’s something we need.

Being an open source tool, it grows through its developer community, which is quite engaged.

It might be a bit daunting at first, due to it being a CLI-based tool, but it is actually extremely fun to navigate around, and once you’ve gotten the hang of it you will surely love it!

Conclusion

These are some our favorite tools, and we’ve given you a brief introduction on them; in the future, we hope to expand on them (ideally all the tools mentioned here, and more!) – if that doesn’t prove to be possible for us, we hope that we’ve at least managed to provide a ‘teaser’ of sorts, and that we’ve managed to pique your interest.

Lastly, honorable mentions go to TinEye – a reverse image search tool, and Phoneinfoga – Python-based phone number scanning tool.