Current Trends in Census Data Capture
With the round of 2010 censuses fast approaching, many countries are preparing for one of the largest logistical exercises they will ever have undertaken. Over 100 countries will be undertaking their national census in the 2 years 2010 to 2011 (source United Nations Statistics Division).
The process of collecting data from every household is a major task for National Statistic Offices (NSOs) and the accurate recording and processing of that data is critical if the censuses are to be of value. Many countries have adopted a face-to-face interview approach, using staff in the field called enumerators. Enumerators visit households in turn and collect and record the relevant census data.
NSOs are under increasing pressure, not only to collect the data in the field, but also to validate and disseminate accurate data in a timely fashion. NSOs are therefore actively seeking technical solutions to help them with this process. Whilst it is generally accepted that a single technical solution will not fulfil every aspect of a national census, it might speed up the process and reduce the costs associated with most of the tasks involved.
The following methods are currently deployed for censuses: Manual Data Entry, Optical Mark Recognition (OMR), Intelligent Mark Recognition (ICR), Personal Digital Assistants (PDAs) and Multi-Channel. Manual Data Entry, OMR and ICR are explained here.
PDAs are used to collect data in an electronic form. They also may have Global Positioning system (GPS) technology incorporated.
Multi-Channel refers to multiple ways of capturing the census data. For example: a statistics office might publish an on-line form on a website and simultaneously setup an automated telephone system for people to call and enter their data through options on a telephone keypad.
Historical View
In recent years DRS has been attending census events around the world, presenting papers and sharing our expertise. In doing so we have met with the staff responsible for managing censuses in many countries and we now hold a view of how technology is being implemented in the global census arena.
Based on recent research conducted by DRS meeting with 51 countries, 30 countries chose Manual Data Entry, 11 countries chose ICR, 7 chose OMR, 2 countries chose PDAs and 1 country chose a Multi-Channel solution for their census data capture.
Of the countries which chose Manual Data Entry, IMPS and CS-Pro software were the most popular software applications used. This software is provided to NSO’s free by the US Census Bureau.
From our sample, paper-based scanning accounted for approximately 36% of the data processing exercise using combinations of OMR or ICR technologies.
Interestingly, we found that how smoothly a census exercise ran was less attributable to the method chosen and more attributable to how well the method was implemented and managed.
Future View
Based on a similar sample of 49 Countries, we can see a significant shift away from manual data collection: 15 countries are proposing to use ICR, 10 countries propose to use OMR, 6 countries propose to use PDAs, 4 countries propose to use manual entry and 2 countries propose to use a Multi-channel solution for their next census data capture. The remaining 12 countries are as yet undecided.
As well as a migration away from manually data entry, there are regional influences on how census data is captured and processed. An example of this is the anticipated use of PDA technology in the South Pacific, The Middle East and South America. Small countries may form regional groups, sharing the same methods or may feel peer-pressure from neighbouring countries to adopt the same technologies.
For the 2020 round of census it is likely that PDA technology will become the most common form of data collection due to the development of technical infrastructures, more readily available technology and reduced costs of the devices themselves. For the 2010 round of censuses this technology is likely to be trialled extensively.
Of the sample of 49 Countries, 12 (approximately 25%) have not decided which method to deploy for their next census exercise. The reasons for this include uncertainties around budget and incomplete tests and trials of the technical solutions.
Those countries that have chosen are mixed in the methods they have selected. OMR and ICR feature strongly and this means that paper-based data capture will play a significant part. Paper is still a popular way of collecting data - despite the large numbers of forms to be completed, the costs are still less than any hand-held digital solution.
Some regions are looking to implement hand-held solutions with a ‘share scheme’ to off-set some of the capital outlay of such large numbers of digital devices with other projects. It will be interesting to so see how this works in practice.
There are also technical limitations to be considered. With ICR for example, the language used may dictate the level of accuracy which can be achieved. Specifically the ICR recognition of some Chinese characters does not yet work well enough to be used for census data capture.
With IT infrastructures becoming more readily available to the general population of many countries, we predict that during the 2020 round of censuses more NSO’s will offer Multi-Channel methods of collecting census data. The internet will doubtless play a large part if there is to be a shift away from face-to-face data collection. Again, for 2010 it is likely that many countries will trial multichannel methods.
Looking beyond 2020, it is likely that census data will be captured on an ongoing basis via other exercises and processes probably through local government. As government departments within countries become more co-ordinated and information is shared more freely, the need for large census data collection exercises could disappear.
published in September 2008