An algorithm for local geoparsing of microtext

Abstract

The location of the author of a social media message is not invariably the same as the location that the author writes about in the message. In applications that mine these messages for information such as tracking news, political events or responding to disasters, it is the geographic content of the message rather than the location of the author that is important. To this end, we present a method to geo-parse the short, informal messages known as microtext. Our preliminary investigation has shown that many microtext messages contain place references that are abbreviated, misspelled, or highly localized. These references are missed by standard geo-parsers. Our geo-parser is built to find such references. It uses Natural Language Processing methods to identify references to streets and addresses, buildings and urban spaces, and toponyms, and place acronyms and abbreviations. It combines heuristics, open-source Named Entity Recognition software, and machine learning techniques. Our primary data consisted of Twitter messages sent immediately following the February 2011 earthquake in Christchurch, New Zealand. The algorithm identified location in the data sample, Twitter messages, giving an F statistic of 0.85 for streets, 0.86 for buildings, 0.96 for toponyms, and 0.88 for place abbreviations, with a combined average F of 0.90 for identifying places. The same data run through a geo-parsing standard, Yahoo! Placemaker, yielded an F statistic of zero for streets and buildings (because Placemaker is designed to find neither streets nor buildings), and an F of 0.67 for toponyms.

Appendices

Appendix 1 Geoparsing for location abbreviations

1.1 Preparation steps

External resource preparation

Download and make available for processing:

• Twittonary

• Dictionary (supplemented with standard technical abbreviations such as tv and iphone)

• Excerpt from National Geospatial-Intelligence Agency gazetteer for the domain (here, for New Zealand)

• List of geographical features

• List of building terms

• List of prepositions

• Location indicators for direction: (N S E W, NE, NW, north, northeast, northwest, etc.)

• Location indicators for distance: (mi, mile, km, kilometer, etc.)

Data preparation

• 1. Normalize tweet: tokenize (word per word), remove articles (a, an, the), and remove punctuation from beginning and end of word and at the end of a sentence.

• 2. Part of speech processing for tweets using third-party, open source software¹⁹

• 3. Find date and time of tweet creation

1.2 Processing steps

Identify place abbreviations and acronyms using heuristics that specify how to match external resources to data. Consider a text word to be a place abbreviation if it:

• * has between 2 and 6 characters and is not in Dictionary or Twittonary is preceded by preposition or direction or distance term

• * matches to a confirmed place abbreviation (that is, an abbreviation that matches the heuristic above

• * matches with an abbreviation known to be a place, either from location abbreviations in Abbreviations.com or from an earlier pass over the tweets

But skip as an abbreviation if it is preceded by # or @

Identify candidate disambiguation words

Data for candidate disambiguation words and phrases are drawn from tweets posted 1 to 5 days before the date of the tweet with the corresponding abbreviation. The disambiguation words and phrases are selected according to the following heuristics:

• first word begins with the same letter as the abbreviation

• if an abbreviation has n letters, take long multi-words of length n-2 words, n-1 words, and n words, and n with the addition of up to 2 stop words

• no verbs

• no hashtags or @mentions

• no colon, semi-colon, question mark or exclamation point within the phrase

Attributes of abbreviations and acronyms and their expansions

Based on inspection of the data, we have arrived at the following attributes that characterize a match between location abbreviation or acronym, and candidate expansion word or phrase. We defined these attributes as:

Create a decision tree model based on attributes

Decision tree algorithms classify unseen data based on data they have previously learned. Given a set of attributes and how these attributes match to the correct classes (and mis-match to other classes), the algorithm will discover how unseen data are classified based on their attributes. The algorithm maximizes the information gain at each decision tree branch test, and makes a model based on the training data. Tests are decided upon during the training phase on the basis of entropy, or the measure of disorder of the data. The model can be considered as a branching set of decisions, or tests. Each test decision branches into a subtree until it reaches a leaf end node. Unseen data is sent through the tree by undergoing a series of binary tests until it reaches a leaf.

Weka’s J48 is a version of an earlier algorithm developed by J. Ross Quinlan, the popular C4.5. We select the J48 option to “prune”, or simplify results. Pruning operates on the decision tree while it is being induced. It works by compressing a parent node and child nodes into a single node whenever a split is made that yields a child leaf that represents less than a minimum number of examples from the data set. Pruning can be used as a tool to correct for potential overfitting, and so an unpruned tree might perform slightly better than a pruned one [11].

Our tree (Appendix 4) has many attributes, and therefore many nodes. The root node must effectively split the data, and the best split is the one that provides the most information gain. Each split attempts to pare down a set of instances (the actual data) until all have the same classification. Each node is tested to determine whether it has a particular value based on which attributes are represented. The data is then routed accordingly.

Given our training data comprised of abbreviations and acronyms and their correct and incorrect disambiguation words and phrases, the machine learning algorithm uses statistics to assign weights to the different features according to their importance. Our decision tree model with 52 leaves appears in Appendix 4.

Because our model is based on our training data as well as these attributes, another model made with different data would obtain different results. We received 87.9 % accuracy with this model, with a corresponding kappa of 0.748, indicating that the model performs much better than chance (which would produce a Kappa of zero).

Features that are significant in correctly classifying the abbreviation or acronym with its disambiguation expression are shown by their location near the root, and also their recurrence in the tree (possibly with different values in different branches). These are:

Rank abbreviation – expansion pairs

Run each abbreviation along with its potential match file through the decision tree. Weka outputs ranking along with true or false and a corresponding error prediction (1 indicates 100 % confidence that the abbreviation corresponds to the match expansion). We wrote a short script that sorts the true values according to error level. We take the top five disambiguation phrases for the sake of error analysis to see whether the correct disambiguation appears near the top, even if it was not selected first.

Appendix 2 Geoparsing streets, buildings and toponyms

User Input (theoretically; data was pre-collected for this study)

• city and county of the data set, possibly also nearby countries

• that country’s abbreviation and the abbreviation of nearby countries

  • Ex. For our data set on the Christchurch, New Zealand earthquake, the user enters:

    • Christchurch (Chch)

    • New Zealand (NZ), and

    • Australia (Aus)

External resources:

• Gazetteer excerpt for region of inquiry

• List of common words that are also place names to filter the gazetteer (list was manually generated)

• Enhanced building list (http://en.wikipedia.org/wiki/list_of_building_types) minus a few ambiguous words such as “wall” and “place”

Data (tweet) preparation

• 1. Save the original tweets

• 2. Make a copy of the tweets for NLP preparation.

• 3. Remove hashtag that was used to retrieve the data set (and recurs repeatedly throughout) in copy of tweets, and tokenize.

• 4. Remove @mentions and replace by XXX. (This is so that we can preserve the original word count).

• 5. Remove tweets in the copy set of tweets that contain unicode characters

• 6. Run the copy of tweets through spell-check algorithm. Retain words that are mis-spelled as well as those that are corrected, in case the spell check alters what is actually correct.

Data (tweet) processing

• 1. Run original tweets through OpenCalais

  • a. Retain locations (city, country, continent, etc.)

  • b. Retain natural features (mountain, etc.)

  • c. Retain facilities

  • d. Ignore all other entities found by OpenCalais

• 2. Find toponyms

  • a. Make all data lower case

  • b. Match against our own gazetteer

  • c. Do not include partial matches (Example: do not take Eiffel if it needs Tower)

  • d. Do not include toponyms in @mentions

  • e. Allow matches when a space is missing between the two words (newzealand)

• 3. Find buildings

  • a. Look at each word in the tweet individually (tokenize)

  • b. Match against building list to find additional facilities

  • c. If building word is found, take two words before and capture the string as output. Also, hard code pairs such as “X and Y buildings”, “X or Y buildings” and multi-word building names such as “W X and Y Z buildings” ²⁰

  • d. To filter non-specific buildings, we do not take those preceded by the article “a”, possessive pronouns “I, my, mine, our, his, hers, yours, theirs”, or relative pronouns “which, what” or demonstrative pronouns “this, that”

  • e. Do not identify as a building if words are found across punctuation mark of period, comma, semi-colon, brackets, parentheses

  • f. Do not identify as a building word, even if it matches an entity named on the buildings list, if it is the first word of a tweet.

  • g. Do not identify as a building if the building phrase contains a placeholder “XXX”

• 4. Find streets within the tweets

  • a. Street identification words: st, street, ln, lane, dr, drive, boulevard, blvd, road, rd, avenue, ave, pl, way, wy

  • b. Check for an Arabic numeral two to three spaces before the street identification word. If a number is found, mine everything from the number to the street identification word. If there is no number, only take the word immediately before the street identification word. Take 3 words ahead of the street word (including a number) plus the street indicator

  • c. To filter possibility of “st” meaning saint instead of street, we use a list of saints’ names.²¹ Matches with the list indicate that the phrase is not a street. Future versions of the algorithm should rely also on word order.

  • d. Do not identify as a street if street phrase is found across punctuation: period, comma, semi-colon, brackets, parentheses

    • Example “14 East. Street that is a dead end”

      Do not mine this as 14 East Street

• 5. Output: tweet—location

Appendix 3 Samples of training data for the abbreviation and acronym classifier

Actual abbreviations and acronyms for location

Park:

Park Avenue

Lex:

Lexington Ave.

Sau Ar:

Saudi Arabia

Zimb:

Zimbabwe

Pac:

Pacific

Papua:

Papua New Guinea

N.A.:

North America

Dom Rep:

Dominican Republic

Mainz:

Mainz am Rhein

SG:

St. Gaullen

SG:

St. Gaul

Qnborough:

Queenborough

Pborough:

Peterborough

L.I. sound:

Long Island Sound

Hunt:

Huntington

N Bay Shore:

North Bay Shore

Jersey Shore:

New Jersey Shore

Ronk:

Ronkonkoma

Rio:

Rio de Janeiro

Kab:

Kabambare

Kago:

Kagoshima

T&T:

Trinidad and Tobago

TT:

Trinidad and Tobago

U.A.E.:

United Arab Emirates

W. Sam:

Western Samoa

Sol. Is.:

Solomon Islands

S.L.:

Sierra Leone

PNG:

Papua New Guinea

SOS:

Southend on Sea

EXM:

Exmouth Gulf Airport

H.H.:

Head of the Harbor

MTA:

Metropolitan Transport Authority

FI:

Fire Island

HH:

Hoek van Holland

Xmas Island:

Christmas Island

Congo:

Democratic Republic of Congo

D.R.C.:

Democratic republic of congo

TdF:

Tierra del Fuego

SP:

Sao Paulo

False Abbreviations & Acronyms

Park:

Parking lot

Lex:

Last expression

Sau Ar:

Sad argument

Zimb:

Zoo in my basement

Pac:

Plenty in cabinet

N.A.:

not available

SG:

Southern Georgia

SG:

so geography

SG:

several grains

L.I. sound:

lighter sound

Hunt:

Hunting

N Bay Shore:

not by the shore

Ronk:

Rings on new keys

Ronk:

Rubber on knives

Ronk:

Recalling our kids

Rio:

Ring in an oval

Rio:

Rinse in oil

Kab:

kneel and bend

Kab:

knots and bends

Kago:

kick and go under

T&T:

Trains and transportation

TT:

tractor trailer

U.A.E.:

Under All Empires

U.A.E.:

Under application employees

Sol. Is.:

Sole Ice

S.L.:

southern languages

PNG:

please not again

SOS:

signs of success

EXM:

expected money

FI:

Finally I

HH:

Hello harry

D.R.C.:

Daily Ritual Cleaning

DRC:

Dr. Classic

DRC:

Drab Rubber Chicken

TdF:

to do Friday

TdF:

Trumpet for December Festival

SP:

sudden park

SP:

spark

SP:

salt and pepper

SP:

sensational paper

Appendix 4 Classifier model for abbreviations and acronyms based on the full training set