A Focus on Data Management; Transforming the way we do Business
By Mark Munsell, Deputy Director, CIO & IT Services Directorate, National Geospatial-Intelligence Agency
Every day, the National Geospatial-Intelligence Agency processes and interprets a massive amount of imagery and other data from aerospace data facilities across the nation. Though this was once our primary data source, the advent of the internet and other notable technological advancements have led to an explosion in exploitable data sources-both classified and open source-and are revolutionizing the way NGA does business.
In order to most effectively and efficiently use these diverse sources, we are evolving our business processes by migrating to the cloud and embracing automation as a means of managing, capturing, and employing this information.
In the past, our predecessor agencies performed imagery analysis on actual film, which had to be manually collected, organized and stored, which was necessarily limited by physical space constraints. With the advent of digital imagery analysis, we were able to greatly enhance our ability to organize, store and retrieve data.
Moving our data to a cloud environment where we can manage and use our data in near real-time is the next logical progression for our GEOINT mission.
Along with enhanced data storage and retrieval, cloud migration gives us scalability to better meet our customers' needs. For example, if a regional crisis dictates a sudden increase in our analytic resource and application use, this scalability can accommodate that shift and facilitate a more rapid response.
However, cloud migration is not the only way NGA is looking toward the future; we are also increasing automation through platforms such as the NSG, the National System for Geospatial Intelligence, Data Analytic Architecture Service, known as NDAAS, to enable processes such as structured observation management, or SOM, and strategies like activity based intelligence, referred to as ABI.
The NDAAS platform, part of NGA's GEOINT Services, is a means of managing big data and actualizing advanced analytics through automated resource management and abstraction. Built on Apache Mesos, NDAAS gives us greater agility in scheduling and scaling while its containerization allows isolation of failed resources. Additionally, incremental expansions and improvements to the service can be deployed rapidly; user load can be scaled up and down quickly without service interruption; and problems can be fixed without taking entire systems offline.
Moving our data to a cloud environment where we can manage and use our data in near real-time is the next logical progression
This means that we are better positioned to quickly meet the needs of our agency, our NSG and Allied System for Geospatial Intelligence partners, and other analysts worldwide.
To help take advantage of these benefits, our data scientists and methodologists are working closely with analysts to build the most result-rich NDAAS applications through a framework that provides a one-stop shop for data science. One where analysts can see, in context, the algorithms they can use and data scientists can help refine those algorithms for greatest efficacy.
Platforms like NDAAS, in turn, provide the foundation for automated analytic processes like structured observation management which is a means by which NGA captures, stores, conditions and organizes data automatically. SOM also enables analysts to disseminate data in real-time, and ensures that our customers receive that data in the same format every time.
For example, when a SOM user enters a specific set of data for analysis in a specific, prescribed manner, the system looks for connections-like similarity of events or geographical proximity-or contradictions between new entries and other ingested data. That preliminary, automated analysis is then available to other users on demand.
In short, SOM's automation finds and reveals immediate connections culled from a big data pool in a clear, standardized way. These connections are used to produce detailed maps and charts that provide regional context as well as geographic reference points, which helps NGA ensure safety of navigation more comprehensively than ever before.
Activity based intelligence is another advanced analytic methodology that helps NGA manage and interpret all potentially viable data sources through automation.
The ABI method asserts that all collected data should be geo-referenced, or indexed, to a point on the earth, which enables discovery of events, entities or activities related to that geo-referenced point. ABI also relies on the principles that all data sources are equally valuable, provided they have spatial correlates, and that forensic data can be as potentially valuable as that which is collected in near real-time, because past data may provide context for current events, even if we did not know the significance of that data at the time of entry.
More importantly, ABI's automated processes make use of all data, not just data in particular "stovepipes." Analysts can use the entirety of this data to better contextualize current events, predict future events, and establish patterns of life for particular areas and actors.
Because this analysis is performed at network speed, ABI also helps NGA more rapidly relay information about actionable activity—not just to other analytic agencies, but to the warfighter and our allies as well.
Though NGA still relies on the data we receive from our aerospace data facilities, we are now looking toward the figurative sky as well.
Cloud migration will not only help us store, organize, and retrieve our satellite data more efficiently but programs that the cloud helps actualize—such as NDAAS, SOM and ABI-will all enable quicker data ingestion, conditioning, and use.
Together, these platforms, processes, and methodologies will help NGA more rapidly and effectively disseminate data, analysis and products to our partners and customers, as well as to our allies and warfighters in the field.
From our satellites in the sky to the "boots on the ground" we support—and all the data in between—NGA is transforming the way we do business as we focus on the future.