In an effort to enable public safety-focused entities to tap social media analytics in emergency response, The National Institute of Standards and Technology (NIST)’s Text Retrieval Conference, or TREC, Incident Streams project intends to gain access to Twitter’s Enterprise-Level application programming interface, or API.

As part of the Public Safety Communications Research (PSCR) program, NIST began TREC Incident Streams project to support research into social media analytics during emergencies and natural disasters. "Increasingly, people expect emergency responders to monitor social media, but in practice, public safety organizations do not have the technology or expertise to do so. The TREC Incident Streams project identifies tweets sent during emergencies and annotates them for priority and according to an emergency response ontology. NIST then invites research teams from anywhere in the world to develop software systems to automatically filter and classify the tweets. NIST develops the infrastructure to measure the effectiveness of those systems, and a forum for sharing research results, which speeds the improvement of the technology. When the technology matures, such systems could drive analytic dashboards or decision-making processes in public safety organizations," says a public notice. 

NIST adds, "Conventional public access to Twitter tweets are limited to small volumes within short periods of time."

NIST requires access and use on an Enterprise-Level API to search Twitter during emergencies, such as safety emergencies like floods, wildfires, earthquakes, other natural disasters, so that we can identify as many tweets coming from the emergency as possible. The vast majority of tweets associated with an emergency with a hashtag are wishes of support and discussion about the event. "However, on a much smaller scale, people in the emergency itself use Twitter to request aid, share resources, offer services, and provide first-hand reports of the situation. NIST’s current practice uses public APIs with multiple queries, but we will be much more effective at collecting all the “critical” tweets during an emergency if we used enterprise APIs, which have much higher limits."

According to the public notice, emergency events identified for collection will have the following properties:

• Have significant duration such that NIST can reasonably imagine social media being an effective communication tool during the emergency
• Involve public safety response including police, fire, and/or emergency health
• Occur in places where English is a primary or significant language
• Not represent civil actions or members of the public exercising their civil rights
• Not be of national security interest beyond conventional emergency management and public safety
• Not arise primarily because of political controversy

Demonstrations, protests, labor strikes, riots and/or large public events without an emergency safety component (like the Super Bowl or a major outdoor concert) are examples of events the TREC Incident Streams track will not collect. 

Twitter offers commercial Enterprise Level access to APIs that allow large scale search and filtering over all Twitter data, says NIST. 

NIST will obtain samples of tweets that people send during emergency events such as wildfires, earthquakes, and hurricanes, which then will be used to measure the effectiveness of technology designed to filter social media streams during those emergency events and route actionable information to public safety personnel.

"The public APIs are very limited in scope and only allow a small amount of data to be examined," adds NIST. "As a result, the datasets we built were small and not completely reflective of the real-world task of monitoring social media from a public safety command-and-control center. Having access to Twitter's Enterprise APIs would give us the opportunity to dramatically improve the quality of data we can provide."