Launching Earth Index Deep Search for comprehensive planetary data

You can now train lightweight classifier models directly on the web

Deep Search is a new powerful search mode, making capabilities that previously required a local machine learning environment set up, immediately available in the Earth Index web application.

Deep Search utilizes a classifier model for detections over a large scale with high accuracy, to comprehensively understand specific environmental changes. Deep Search classifier models have been critical for large scale searches covering tens of thousands of locations, like locating global cattle facilities to model emissions with Climate TRACE, and land use change to identify disease vectors with Stanford.

All Earth Index Full Access users now have Deep Search enabled. If you are on the Open tier, request Full Access directly in Earth Index. If you are new to Earth Index, sign up now at app.earthindex.ai.

The distinction between Quick Search and Deep Search

This mode complements Quick Search, which uses nearest neighbor search in embedding space to almost instantly surface predictions. This is essentially a sorting mechanism that averages across positive and negative labels, and takes the difference as a basis for the search. This is very fast and can yield good results, but it is not exhaustive. 

Deep Search runs are more powerful, requiring minutes to fully train a classifier. Technically, Deep Search trains a binary classifier on positive and negative labeled embeddings, and runs the resulting classifier on every single part of the project’s area. This is more computationally intensive, accounting for the longer run time (more on the engineering that made this happen in a future post). The result is a comprehensive evaluation of the entire geographic area and can return over 100,000 results - 100 times more than QuickSearch!

Each of these approaches is valuable on its own but they really shine when used together. We've seen great results when users start with QuickSearch, test the viability of the search, quickly build up a small (<100) set of points, then use those in DeepSearch to find the rest.

We’ve worked hard to deploy these capabilities efficiently. Just last week, Deep Search got a thorough exercise at Data Harvest, where our partners at Pulitzer Center shared Earth Index training with investigative journalists. 50 participants searched together and our database, API, and modeling infrastructure scaled and didn’t break a sweat. We saw similar strong performance during the Africa Mining Watch Mapathon in April.

Learn more with us

We’ll be hosting events and office hours in the next few weeks where we’ll demo how to use Deep Search and answer your questions. Look out for more details soon.

Our goal with Earth Index is to get powerful tools in the hands of those closest to environmental challenges, reducing the technical and resource barriers of processing satellite data and running models. Deep Search on the web is a unique capability for Earth AI, and we’re eager to see what you do with it. Try Earth Index now at app.earthindex.ai.