U.S. Department of Justice Office of Justice Programs Bureau of Justice Statistics Assessing Measurement Techniques for Identifying Race, Ethnicity, and Gender: Observation-Based Data Collection in Airports and at Immigration Checkpoints January 2003, NCJ 196855 -------------------------------------------------------------- This file is text only without graphics and many of the tables. A Zip archive of the tables in this report in spreadsheet format (.wk1) and the full report including tables and graphics in .pdf format are available from: http://www.ojp.usdoj.gov/bjs/abstract/amtireg.htm --------------------------------------------------------------- Based on a report and data provided by Humanalysis, Inc. Over the past several years State and local governments have engaged in data collection regarding demographic characteristics of persons stopped by the police. These efforts are aimed at understanding factors used by law enforcement to make such stops. While data collections on law enforcement encounters have been undertaken, analysts have debated the availability of methods to meaningfully analyze these data. One complicating factor is the need to identify the baseline data necessary to make assessments regarding different racial groups' experience with the police. In 1999 several Federal law enforcement agencies designed and implemented data collection procedures to capture race and ethnicity information on persons stopped by their officers. The Immigration and Naturalization Service's (INS) Border Patrol agents began collecting race and ethnicity data for those persons stopped at selected border crossings and highway checkpoints. Likewise Drug Enforcement Administration (DEA) agents collected data on nonspecific suspects stopped in selected airports. At the State level, studies have been undertaken in Maryland, California, and New Jersey. New Jersey measured characteristics of persons using the New Jersey Turnpike and those that were speeding. (See list of sources.) The various Federal and State agencies collecting the data wanted to have baseline data available to estimate the race and ethnicity of all persons passing through their area of responsibility. Lacking this kind of baseline data, it would not be possible to know whether the characteristics of the persons stopped were disproportionate to all those who had a probability of being stopped. Scope The Bureau of Justice Statistics (BJS), working with the Bureau of Transportation Statistics (BTS), contracted with Humanalysis, Inc., of Orlando, Florida, to conduct observational studies at two sites: 1) Immigration and Naturalization Service (INS) Border Patrol Checkpoint along Interstate 5 in San Clemente, California, and 2) the Detroit Metropolitan Airport. This report describes the findings from both data collections and assesses the difficulties in implementing this kind of a study. The primary object of the observational studies was to determine the feasibility of using such techniques for estimating the demographic characteristics of persons coming through the checkpoint and airport, and what issues would be involved in replicating this technique in other locations. Can people's race and ethnicity be easily recorded from observational techniques? To what extent do practical issues, such as gaining authorized access or proximity to persons under observation, play in the implementation of this kind of study? OMB standards for classification of Federal data on race and ethnicity OMB Statistical Policy Directive No. 15 "Race and Ethnic Standards for Federal Statistics and Administrative Reporting" established standards for observer-collected data on race and ethnicity. Federal law enforcement agencies participating in the data collection used these categories on their data collection forms. The base data collection at the two sites implemented the combined race/ethnicity format which uses the following categories: * American Indian or Alaska Native * Asian * Black or African American * Hispanic or Latino * Native Hawaiian or Other Pacific Islander * White. --------------------------------------------- This study was supported by Bureau of Transportation Statistics contract number DTTS59-01-F-10151 to Humanalysis, Inc. The contents of this document do not necessarily reflect the views or policies of the Bureau of Justice Statistics, the U.S. Department of Justice, the Bureau of Transportation Statistics, or the U.S. Department of Transportation. ---------------------------------------------- Practical issues for observational studies Practical arrangements were necessary to conduct an observational study, such as gaining permission to access the site and finding an unobtrusive location within the site, as well as dealing with issues of traffic volume, poor lighting, and multiple points of ingress and egress. Site access Gaining approval to conduct the observation test project required a great deal of effort over many months. BJS was denied access by three airports before the Detroit airport was approached and agreed to allow observers into the facility. Numerous sites were reviewed before the INS San Clemente checkpoint was selected. The terrorist attacks of September 11, 2001, also added a 4-month delay in obtaining permission to conduct the studies. Limitations Demographic data were not available from either site. Since it was not critical that the observer accurately categorize the person's race or ethnicity, as the premise of racial profiling is the perception of the officer, the project tested the observation method to assess whether baseline data could practicably be collected. Border crossing checkpoint study Study location INS selected the Border Patrol checkpoint at San Clemente, California, as the site for the observational study. The checkpoint is located on Interstate I-5 approximately 5 miles south of San Clemente, within the Camp Pendleton US Marine base. Interstate I-5 is the main north-south coastal highway in California and has four traffic lanes at the checkpoint with daytime traffic flow normally in thousands of cars per hour. The Border Patrol site has a two-story operations building and a one-story administration building for the patrol agent in charge and staff. Border Patrol checkpoint on I-5 south The actual checkpoint was covered by a pavilion that shades the area during the day. Suspended lights provided illumination after sunset. The volume of traffic at the checkpoint varied with time and day. The Border Patrol monitored traffic flow using video cameras south of the checkpoint. When the backup of vehicles extended more than a mile, the checkpoint was shut down and vehicles were allowed to pass at highway speed. This occurred most often during the morning and afternoon rush hours on weekdays and from midmorning through evening on weekends. Setting up the checkpoint consisted of positioning vans in both breakdown lanes, rolling out two "wheeled" stop signs, and opening the circuit breaker controlling power to the checkpoint area. The vans had two large, flashing red lights on the front to signal traffic to prepare to stop and a series of bright lights along the side that shone into the cars as they came to a stop. This additional lighting was particularly helpful at night, as the overhead lights of the checkpoint structure created deep shadows inside the cars. The Border Patrol agents positioned themselves behind the mobile stop signs between lanes one and two and lanes three and four. Each agent controlled the two lanes passing on either side. The vehicles were waved through the checkpoint at 5 to 10 miles per hour until an agent decided to stop one. While agents looked for vehicles carrying anyone they might recognize as a felon (a "most wanted" gallery was posted in the operations anteroom), they also stopped any vehicle with "covered or occluded objects in the backseat (or elsewhere)." Most stops resulted in a quick survey of the interior and a few verbal questions and the vehicle was allowed to continue. An agent might require a vehicle to pull forward to a covered inspection area to the right of the highway. There, additional agents assisted in inspecting the vehicle and questioning the occupants. Dogs were also used in the area to sniff out drugs and other substances. When criminal activity was suspected, the suspects were handcuffed and removed from the area. Viewing location at checkpoint From the border checkpoint location on I-5 the enumerators were able to see the nearest lane of traffic with a downward viewing angle of approximately 18 degrees. The distance from enumerator to car was approximately 20 feet. Two obstacles limited the viewing range: the heavy support girders, both vertical and diagonal, of the checkpoint and the Border Patrol van parked in the right breakdown lane at the stop point. The enumerators had an effective viewing range, left to right, of approximately 54 degrees. The distance between vertical supports was approximately 9 feet. Enumerators Four enumerators (two white females and two white males) rotated through observation sessions in pairs. All enumerators were required to wear Border Patrol identification badges during their time on the site. Materials All data were recorded on a survey collection form containing columns for date, time, vehicle identification, window tint, driver and passenger race/ethnicity and gender classification, as well as indicators for observational certainty. As specified, the possible race/ethnicity classifications were in accordance with the OMB standards. An additional classification of "unknown" was also available. Observation sessions Day one (Thursday, January 10, 2002) The first pair of observers, a male and a female, conducted three sessions on the first day of the study. The observation times were spread out during morning, afternoon, and early evening. In the first session it became apparent that the limited viewing range, coupled with the speed of the vehicles during normal operation, impacted the observers' ability to collect all data on the survey collection form. The observers could not see license plates, and the only vehicle identification possible was an occasional notation by both enumerators such as "red car" or "blue SUV." A slight break in traffic flow (three car lengths or more) gave the opportunity to record the type of vehicle approaching. These became the only milestones for assuring the alignment of collected data. Following the first 5 minutes of the first recording session, the fields (L, M, H for low, medium, high surety) on the collection form for recording the surety of each assignment of race/ethnicity were also ignored. There was insufficient time to record any data other than the race/ethnicity and gender of the driver and of any passengers. Due to the pace and volume of traffic during the first session, subsequent observation sessions were limited to 30 minutes to maintain the quality of the recordings. Day two (Friday, January 12, 2002) On the second day, two additional enumerators were trained during the first morning session. They were then paired with an enumerator from the previous day for sessions 5 through 8. Due to heavy traffic flow during late Friday afternoon, the checkpoint was shut down. During the evening the enumerators experienced difficulty viewing into the vehicles when the area lights were illuminated. As a consequence, the evening sessions were stopped for the day. This experience indicated that the use of enumerators to record data on passing vehicles at night would not be productive. This phenomenon could apply to controlled land border crossings as well. Day three (Saturday, January 13, 2002) On day three of the study, four sessions were completed which included stoppages for checkpoint shutdowns. The last session of the day occurred just before noon. The traffic flow was so heavy the watch commander determined that the checkpoint would be shut down for the rest of the afternoon. Results All observational data for each session and by each enumerator were compiled, resulting in 12 sessions that represented 3,534 pairs of vehicle observations. The race/ethnicity and gender classifications were recoded into numeric values for further analyses. The frequency and percent distributions of the total driver observations for each classification were summarized for each session. The enumerators classified the majority of drivers across all sessions as "White male" (ranging from 35% to 60% of all observations. The enumerators classified more drivers as "White female," than other nonwhite male or female classifications, across seven sessions (ranging from 17% to 23% of all observations). During five sessions -- 4, 6, 8, 9, and 10 -- at least one enumerator in each session classified more drivers as "Hispanic male" than "White female." Across all sessions, no drivers or passengers were classified as "Native Hawaiian/Pacific Islander" or "American Indian/Alaska Native." The enumerators classified a majority of vehicles across all sessions as having only a driver with no passengers. The percent of driver-only vehicle observations ranged from 60% of the total weekend observations to 96% of the weekday observations. Across the majority of reported observations of drivers with passengers, the race/ethnicity classification of the driver matched the race/ethnicity classification of the passenger(s). The percent of race/ethnicity observed matches of drivers and passengers ranged from 80% to 100% of all combined observations. Discussion The degree of inter-rater agreement for driver race/ethnicity classification was relatively high across all sessions. Combined, the paired enumerators agreed on the race and ethnicity categorization of observed persons in 77% of the cases. The checkpoint study was limited by the constraints of the enumerators' position at the site location and the continuous flow of the traffic being observed. Detroit (Wayne County) Metropolitan Airport Study location The Department of Justice selected the Detroit Metropolitan Airport for the observational study. Observation locations and collection times were restricted by the airport's legal counsel in consultation with the Detroit Airport Federal Aviation Administration (FAA) security chief. Contract personnel were limited to observations in unsecured areas of three terminals between 10:00 a.m. and 8:00 p.m. Viewing location at airport Project staff initially visited all possible viewing locations in the Detroit airport's unsecured areas. Ultimately, two clear viewing locations were found for data collection. One observation point was in the Northwest Airlines main terminal. The terminal consists of two large ticketing areas (about 12 ticket agent positions per area) positioned on either side of the access to the security point. The Northwest Airlines terminal manager restricted the enumerators to a standing position against a wall approximately 6 feet from the roped serpentine enclosure that guided passengers into the security checkpoint. The second observation point was in the Northwest International departure. Only one observation session was conducted at this location at a departure time when the passenger traffic was substantial enough to collect an adequate sample. The viewing location was again within 6 feet of the roped passenger enclosure. Sessions were limited to 30 minutes to maintain recording quality. Enumerators Two enumerators (white female and white male) conducted all observation sessions. While arrangements had been made with local universities for additional enumerators, the limitations placed on viewing locations eliminated the need for additional personnel. Materials All data were recorded on a survey collection form containing columns for date, time, sampling location, passenger race/ethnicity, gender, age (adult/ child), type and number of carry-ons, whether she/he was traveling alone, and an indicator of observational certainty. The race/ethnicity classifications were in accordance with the OMB government standards. An additional classification of "unknown" was also used. Observation sessions (Thursday, January 31, 2002, through Saturday, February 2, 2002) The enumerators conducted four sessions on the first day of the study. Due to adverse winter weather conditions, more than half of the day's flights were canceled and passenger traffic through the security checkpoints was sporadic. As weather conditions improved, the enumerators conducted 30-minute observation sessions during the morning, afternoon, and early evening. After the first 5 minutes of session 1 observations, it was apparent that the close viewing range made identification certainty "high" and this category was ignored in subsequent recordings. Three observation sessions were conducted in the main terminal and one session was conducted in the international departure terminal. Three additional observation sessions were conducted on each of the second and third days in the main terminal. Results All observational data for each session and each enumerator were compiled resulting in 10 sessions representing 1,928 pairs of departing passenger observations. The gender, race/ethnicity, and carry-on classifications were recoded into numeric values for further analyses. The frequencies and percents of the total observations for each race/ethnicity and gender classification made by each observer were summarized for each session. The majority of departing passengers observed in all 10 sessions were classified as white (from approximately 78% to 94% of the total observations). The majority of departing passengers across 9 of the 10 sessions were white males (from approximately 41% to 58% of the total observations. White females were observed more frequently than other male or female classifications across all 10 sessions (from approximately 33% to 49% of the total observations). More white females than white males were observed during session 8. Across all sessions, only two passengers were classified as American Indian/Alaska Native, and none was classified as Native Hawaiian/ Pacific Islander. Discussion The 10 sessions conducted over 3 days in this pilot observational study allowed the observer to assess the inter-rater reliability of race/ethnicity and gender identification. Given the proximity of the observers to the passengers, race/ethnicity, gender, and age identifications were fairly easy to determine and the inter-rater agreement was good. In the airport the paired observers agreed with their race and ethnicity categorization of persons in 97% of the cases (1,862/1,928). Recommendations for future observational studies Gaining access to sites and viewing areas with the best proximity for observational studies was the most difficult aspect of this project. To assess the validity of these observational classifications, sessions would also require additional data based on the subjects self-reports of their race/ethnicity. An actual border crossing, where each car is stopped, would be more suitable than a checkpoint. Using such a location would allow interviews of a random sample of drivers concerning their demographic characteristics and a comparison of the results to enumerators' classifications. A comprehensive airport study requires access to all areas (gates, security checkpoints [from both sides], concourses, entrances, and so on). Airports with various designs and open or less restricted access need to be selected for future studies to provide an opportunity to undertake a comparison of sampling designs, and to allow for the collection of demographic data from airport users. Sources for further information Final Report of the New Jersey State Review Team Regarding Allegations of Racial Profiling, July 1999. General Accounting Office (GAO). Racial profiling: Limited data available on motorist stops. March 2000. (Report to the Honorable James E. Clyburn, Chairman, Congressional Black Caucus.) Harris, David. "Stories, the Statistics, and the Law: 'Why Driving While Black' Matters, Minnesota Law Review. Volume 84, Issue 2. December 1999: 265-325. Lamberth, John. Revised Statistical Analysis of the Incidence of Police Stops and Arrests of Black Drivers/Travelers on the New Jersey Turnpike Between Exits or Interchanges 1 and 3 from the Years 1988 through 1991. November 11, 1994. . Montgomery County, Maryland Department of Police. Traffic Stop Data Collection Analysis: First Report Covering the Period October 2000 through March 2001. October 31, 2001. -----. Traffic Stop Data Collection Analysis: Second Report Covering the Period April 2001 through September 2001. January 30, 2002. -----. Traffic Stop Data Collection Analysis: Third Report Covering the Period October 2001 through March 2002. May 31, 2002. Office of the Attorney General. Speed Violation Survey of the New Jersey Turnpike: Final Report. Trenton, NJ. December 2001. San Diego Police Department. Vehicle Stop Study: Mid-Year Report. September 21, 2000. San Jose Police Department. Vehicle Stop Demographic Study. December 1, 2000. (for the period June 1999 to July 2000). ------------------------------------------ The Bureau of Justice Statistics is the statistical agency of the U.S. Department of Justice. Lawrence A. Greenfeld is director. Humanalysis Inc. collected the data and provided the information for this report. At BJS, Steven K. Smith and Carol J. DeFrances prepared this report. Carolyn C. Williams produced and edited the report. The following individuals provided assistance to Humanalysis personnel as they conducted the border checkpoint and airport observational studies: James St. Hilaire from INS, Stephen Gonzales from INS, Wes Knippler from INS, and Delphine Fairbanks from the Detroit Municipal Airport. January 2003, NCJ 196855 --------------------------------------------- End of file 01/23/03 ih