Python Tutorial: Using the Census API

Показать описание

---

The data frame used in the first lesson was downloaded from the Census API server. Now we'll learn how to construct an API request and load the response into a pandas data frame.

This is a basic Census API request.

The part up to the question mark is referred to as the "base URL", and specifies, the host, year, and dataset.

The part after the question mark is referred to as the query string.
This is where we specify parameters such as the variables being requested, and the desired geography. Although the URL can be constructed using simple string manipulation...

...we will use the requests library, imported here, to construct the URL.

We define variables for the HOST, year, and dataset,

then build the base URL by joining these variables with a slash. dataset = "dec/sf1" stands for "Summary File 1", and refers to the full count data from the decennial census.

The variables to request are assigned to a list, get_vars. These include the name of the geographic unit, the land area in square meters, and the full population count. Don't worry about the obscure name "P001001". We'll learn where to find these variable names later.

A dictionary key "get" is created by joining the variable names into a comma-separated string.

Set the "for" key to the geographic level. We use "state", followed by the asterisk wild card, to request all states.

Finally, execute the request and store the return value in the response object "r".

Now inspect the text attribute of the response object.

This is a string with the form of a list of lists. Each sublist is a "row" of data, with the first row being the table header. Notice that all values, including numerics, are double-quoted. When we load this data into pandas, we will have to fix these data types.

In case of a poorly formed request, the API server will return an error message.

One common error is specifying the variable name incorrectly.

The json() method of the response object returns a list of lists.

The first sublist contains the column names. Every API call returns geographic identifiers. This one returned a column named "state".

Throughout this course, we will be renaming columns to make them easier to work with.

Begin by creating a list containing the new column names, all lower case.

Now construct the data frame using pd.DataFrame. The new column names get passed to the columns parameter. The json list of lists gets passed to the data parameter. Use slicing to skip row 0, which contains the header.

Two columns have integer data currently stored as text. Fix the data types by passing the correct type, int, to the astype method.

Use head to view the result.

The "state" identifier, like a ZIP code, has leading zeroes and should be left as text.

Now that the data is loaded, let's see an example analysis.

Create a new column "pop_per_km2" by dividing the population by the land area. The result is in square meters. Multiply by 1000 squared to get the density in square kilometers.

Then use nlargest to return the 3 top records. The first parameter is the number of rows to return, the second is the column to sort by.

You've seen some key elements of the Census API. Now it's your turn to try some examples.

#DataCamp #PythonTutorial #AnalyzingUSCensusDatainPython #UsingtheCensusAPI #AnalyzeUScensusdata