System-to-System Data Collection in business surveys applied to an agricultural survey: a Proof of Concept

On past October 22nd, my former colleague Ger Snijkers presented "System-to-System Data Collection in business surveys applied to an agricultural survey: a Proof of Concept" at the "UNECE Expert Meeting on Statistical Data Collection 2022".

After leaving Statistics Netherlands, it's really nice to see my old data collection project moving forward. It's in very good hands.

Authors: Ger Snijkers, José Gómez Pérez, and Tim de Jong 

Abstract:

At the end of the 2018 edition of this workshop, I pitched the idea of automated data collection for business surveys. In 2019, I presented the results of a first exploratory study. Now, 3 years later we have a Proof of Concept.

In the 20th century, sample surveys have proven to be a cost-efficient method to produce accurate statistics, although they come with a high cost both for the National Statistical Institutes (NSIs) and businesses, who may experience high response burden. Nowadays in the information age, there are a lot of new digital data sources in smart industries, like in precision farming. In some cases, these data sources allow for data communication with other computer systems without human intervention via Application Programming Interfaces (APIs). Based on these software interfaces, we developed a system-to-system data collection methodology that reduces response burden by automating the business’s internal data retrieval process. Applied to the official Crop Yield Survey, a software prototype was developed based on this methodology.

At the workshop, we will present the IT architecture we developed, showing how data capture and processing can be automated. We will discuss the automated pre-filling of the electronic Crop Yield Survey questionnaire using an API provided by a smart farming machine manufacturer, John Deere: the MyJohnDeere API. In a first Proof of Concept, it has been applied to data from a virtual farm, showing that it works, and the farmer’s workload to complete a questionnaire can be limited to a minimum.

Our next step is to conduct a small-scale field test with a small number of farmers to study the method in practice. This field test is planned for the fall of 2022. Hopefully, we can present the first results of this field test at this Expert Meeting in October.

It is our belief that this system-to-system method can be applied to business surveys in general and in the future will replace the manual completion of business survey questionnaires including the manual retrieval and rekeying of data.

The slides can be found here: https://unece.org/sites/default/files/2022-10/DC2022_S2_Netherlands_Snijkers%20et%20al_P_2.pdf