Agricompas and Information Workers working in a big data project for the Agro-Industry

Agricompas and Information Workers working in a big data project for the Agro-Industry

by Carlos Torres, Senior Consultant, IWCO

Information Workers, or IWCO, is an important technical partner working on the EcoProMIS project led by Agricompas.

A Colombian company with 10 years of experience, IWCO is focused on helping its clients get the most and the best value from their data by turning it into information that allows them to make decisions in the short, medium, and long term.

This value comes from four fundamental areas in the process of data harnessing: exploration, extraction, refinement, and consumption of data.

Data Exploration
This phase is designed to understand the types of data, their characteristics, where they are located and their potential value, additionally, the identification of business needs that can be solved with data.

Data Extraction
During the extraction process, we help our clients (such as the EcoProMIS project) to take their data, put it into a structured format that can be simple to use and that enables our clients to immediately create on-demand queries to get answers to their business questions.

Data Refinement
Once the data is in a structured format, we help our clients design and create models to predict, understand, and even extract hidden patterns in the data.

Data Consumption
Business users such as Agricompas and Pixalytics are extremely important, that is why we care about helping them consume their data models – information that can now be prepared for analysis. Through knowledge transfer processes we enable our clients to obtain answers to their strategic questions.

Information Workers is a Microsoft, AWS (Amazon), and Google Partner. With broad experience and a process focused on culture, we have an emphasis on helping organisations to operate in the world of self-service.

We have a multidisciplinary team that includes Mathematicians, Statisticians, Economists, Software Engineers, Systems Engineers, and even Petroleum Engineers (!). This diversity means that our team is able to bring our customers’ different professional perspectives, something that has helped us contribute value in the EcoProMIS consortium of seven international partners.

Together with Agricompas, we created a plan designed to resolve some of the challenges for the EcoProMIS project, namely how we acquire, store, manage, and secure the agricultural analytics data.

After developing this plan, it was essential to choose a platform aligned with these necessities pragmatically, and allow us to obtain results in a short period. Another important step was to identify the sources of big data for the project, for example publicly available data (e.g. earth observation images from UKSA), streaming data from different sensors like weather stations, and data from legacy databases.

Big Data
As we engaged with all of this data, it became obvious that we could not process this multivariate data using traditional methods. The data from the EcoProMIS project is clearly in the realm of big data, complying with the three ‘V’s:

Volume: EcoProMIS has collected a lot of data from different sources.
Velocity: The various data streams are handled within different timeframes from the different sensors.
Variety: All types of formats – from structured to unstructured.

Adding Value
Another key step in our contribution to the project was to find a way to access, manage, and store the data. We needed to find a data platform that can support the storage and the capabilities of analysing petabyte-size files and trillions of objects.

Equally important was prioritising security and data protection, making sure the database is GDPR-compliant, and that data is securely stored. Linked to this is our role in providing data auditing and ongoing support.

Agro-Industry
Big data in the agro-industry plays an important role. With such a large amount of information out there, the data needs to be shaped or tested in a way that adds value to the agro-industry.

By doing so, the agro-industry can better identify problems and reach the goal of sustainable optimisation that is at the heart of the EcoProMIS project.

New year, new rice season

New year, new rice season

By Sam Adams, Head of Programmes at Agricompas

Over the last month or two the Agricompas science team have been making visits to rice fields in the Tolima region in Colombia. Travelling the five hours from Bogotá, the journey is a reminder of just how big this beautiful country is.

The two scientists are using these trips to better monitor, understand, and support the collection of accurate crop data. Inspections of soil and water took place, as well as looking at our cutting edge technology that collects weather data as well as greenhouse gas emissions.

These trips are in close collaboration with another member of the project consortium, Fedearroz. Their technicians were on hand to demonstrate data collection techniques and to reflect on best practices and quality control measures.

Dr Gil and Dr Bojacá inspected the rice crop, which is just starting to germinate. This is the fifth cycle of rice crop data that the EcoProMIS project is collecting from both the Tolima and Casanare districts.

The rice seed in Tolima is just starting to germinate

 

In addition to these five cycles of data, we have access to a much larger historical database of rice farming, and our team is combining this information in order to create and strengthen our crop model. This model is thus becoming more reliable (‘robust’) as we add more data, including phenological, weather, and satellite imagery.

As we prepare to enter the fifth year of the project, it is exciting to see how a huge amount of collected data has now been  processed through our science and IT work into useful practical outputs that will directly benefit farmers.

Via mobile apps, these growers are able to access yield calculators and other tools to help them maximise their farm output and move towards our shared goal of sustainable optimisation.

Our science team will continue to make these regular visits as the project grows from strength to strength and we look towards rolling out this unique service to rice growers in Colombia and beyond.

Pilot workshops with rice growers

Pilot workshops with rice growers

By Elizabeth Sweitzer, CIAT

In early December 2020, the EcoProMIS project partners presented workshops to update the status of the project with rice growers across two major regions in Colombia. These workshops were an opportunity to share about EcoProMIS news and technical developments and to get critical feedback from growers about their perceptions.

There were around 16 growers present across the two regions for these events. Our workshops were presented by Gabriel Garces of Fedearroz, although this was a team effort together Agricompas, CIAT, and Solidaridad.

Update of the project and timeline

During the workshops, Gabriel explained the progress of the project from December 2017 until present day. The growers were informed that current tasks include calibrating the advanced knowledge services, testing the technical systems, and getting everything ready so that the platform can be successfully delivery to growers and to the insurance market.

 

Growers were shown how the EcoProMIS app will give them advanced knowledge about their farms

 

 

Four app features

Gabriel went on to describe the four major services on the EcoProMIS platform. The first of these is the visualisation and management of information, where farmers will have a user-friendly interface to assess their best strategies for sustainable optimisation.

Secondly, the platform will provide output predictions based on climate, farm management practices, and data collected on crops. This has been designed with growers and their rice federation, as explained in this blog here.

Thirdly, the EcoProMIS app provides comparative charts to assess a grower’s land in contrast with other land in their area as well as providing a historical comparison. This feature allows growers to benchmark their productivity and will lead to exploring better management practices.

Finally, the workshop showcased crop diagnostics, based on geo-referenced data. With funding from the UK Space Agency, one of our project priorities is to commercialise applicable services from satellite data. Gabriel explained that each field is regularly photographed by satellites and that our platform can take those images to help improve farm productivity.

Further growth

The workshops also took the opportunity to explain that at a later date the project will also be contributing information and discussing insurance schemes with farmers, using anonymous (GDPR-compliant) data to lower premiums and close the ‘insurance gap‘.

Gabriel went on to explain more about how scientists collect geospatial data, and how those data points are turned into tangible information about the health of crops, making estimations about the phenotyping stage of the crops, and more!

Gabriel emphasised that the platform will be totally free for growers. He showed helpful visualisations of what the platform will look like for growers. A couple of screenshots are attached, in Spanish, to this blog.

 

The EcoProMIS app gives growers yield estimates based on their management choices

 

Initial impressions and next steps

The growers present at the workshop were enthusiastic about the power of the app’s knowledge services. In particular, growers expressed interest in the ability to estimate growth and output and the ability to plan for different scenarios, for example, based on the use of different seeds, fertilisers etc.

Growers were also curious to learn more about how the science works. For example, they wondered how does the app calculate output when we consider how variable the weather is with global climate change and phenomena like the recent La Niña? We spoke more about how the app handles this type of data variability, how we develop metrics, measurements and more.

The next steps are to upscale these workshops and deliver more in January and February of 2021. These new workshops will be based off of the workshops we gave in December 2020, incorporating the initial attendee feedback so that the workshops are even more accessible and informative to growers.

 

 

High-tech: an undisputed ally for oil palm cultivation

High-tech: an undisputed ally for oil palm cultivation

By Jorge Torres-León, José Monsalve, Cristian Angarita from Cenipalma

Over the past few decades, huge growth has been shown in geospatial technology applications in different fields worldwide. As for projects in agriculture, there has been a considerable increase in the use of images provided to us by earth observation satellites. These images allow us to obtain a top-down view of very large areas, to analyse the terrain, and to have better data for decision making.

Like all technology, the use of satellite images has both advantages and disadvantages. Although it is possible to have good quality images of almost any location on the planet, its most noticeable disadvantage is that since the sensor is inside a satellite orbiting at very high altitudes, it captures the clouds, thus creating a ‘mask’ above the objects requiring observation. Colombia, being in a tropical area, has a very high density of clouds most of the time, making this a frequent difficulty.

This is where we find the importance and efficiency of using other technology, namely unmanned aerial vehicles (UAVs or drones), since they operate at lower altitudes. As regulated by the Colombian Aeronautics Authority, all UAV flights that we conduct are below 150m and thus drones flying over oil palm crops are able to capture very high resolution images free of clouds.

Drone flights support farmers with decision making

Equipped with multispectral and thermal sensors, our UAVs have collected images in oil palm plantations in different areas of Colombia. Regular flights means that the EcoProMIS project is able to achieve continuous monitoring and comparison of the development of the plants. Different phenological (growth) stages of the crop are observed, including the unproductive, stabilisation, and productive phases.

Through multispectral images, such as those seen below, it is possible to perform calculations using the spectral bands being captured by the sensors. One approach we use is the vegetation index, in which it is possible to remotely detect the status of individual plants in terms of growth, pest infestation, water stress, flooded areas, nutritional deficiencies, systemic diseases, etc.

For the agricultural sector, this type of information is extremely important. The EcoProMIS platform sends such information directly to the grower’s mobile phone, and this near real time data allows farmers to take preventive measures to support crop performance.

Example of data types from our drone flights.

 

Another technique we use is with images acquired by the thermal sensor on the drones, which provide information on the temperature of each of the surfaces within the study area. Using this technology, a farmer can pinpoint problems associated with disease and water stress. The acquisition of spectral images can be scheduled on a daily basis to quickly identify and quantify unhealthy plants, such as those suffering from chlorosis. Early identification allows farmers to make decisions in a timely manner so as to protect their crops and yields.

Connected weather stations

Digitally-connected weather stations are another form of high-tech equipment being used by the EcoProMIS project to support farmers. These have been installed in experimental plots and offer the advantage of knowing in real time the climatic conditions such as rain, temperature, humidity, etc.

It is essential to combine data collected by the weather stations with the data from drone images and oil palm models, as almost 70% of the crop performance can be explained by the surrounding climatic conditions.

Greenhouse gas data

Currently, our project collects GHG data through eddy covariance towers in two different palm regions in the country. This system allows us to quantify the CO2 (carbon dioxide) that is absorbed by the oil palm during its photosynthetic process as well as the emissions of greenhouse gases such as CO2, methane and even water vapor. Through these measurements it is also possible to calculate in detail the evaporation of the ecosystem.

Eddy covariance tower in one of our oil plam fields

 

These high-tech systems allow more precise readings of environmental variables, such as relative humidity, atmospheric temperature, direction and wind speed, precipitation, etc. All these variables are of critical importance in establishing the influence of weather conditions on the development of oil palm cultivation as well as the effect the crop itself has on the environment.

By combining the data from satellites, drones, weather stations, and GHG towers we have a rich and detailed understanding of each farm. When this data is turned into knowledge services (such as yield-prediction) and delivered through a mobile dashboard, growers are supported so that they can make rapid and intelligent decisions in their farm management.

As we at Cenipalma work together with the other project partners in Colombia (CIAT, Fedearroz, IWCO, Solidaridad) and in the UK (Agricompas, Pixalytics) we are contributing to a high-tech digital platform that will be a great ally to oil palm cultivation.

 

Inviting farmers to dance

Inviting farmers to dance

by Carlos Perez, Solidaridad 

A few years ago, a man became famous when someone shared a video of him dancing at a public concert. His spontaneous and unusual exhibition although original was not the only element to become popular on the web.

Analysis of this video provides some insights on how new technologies are adopted and is instructive for how we at EcoProMIS are working with farmers to develop our platform.

 

 

The Dance Grows

What at first seems to be a solo dance, a moment later prompted the participation of a second member. The second dancer’s role was not limited to emulating the pioneering dancer. On the contrary, we can see how number two receives instructions on how to perform the dance, while also providing feedback to the pioneer.

Further evidence of the new arrival’s genuine interest is how he invites others to join this spontaneous duo. Like the second, the third dancer follows the movements of the first dancer, while improvising according to the music and rhythm.

What began with an eccentric loner, in a moment becomes a choreography performed by three dancers. We see how the attention generated in the audience, which was initially low, begins to grow.

Soon, the group increases in size again, this time not just by one more dancer, but about five more, and in the blink of an eye, the group reaches at least a dozen. This sudden growth of the group of dancers triggers the arrival of new members and in less than a couple of minutes it grows even further.

Dance as Metaphor

This video, more than simply being an inspiring dance at a music festival, has been considered as an interesting reference about how a forerunner or pioneer (a particular product or technology) can trigger a crowd action.

Like dancer number two and three, we see that there are minority groups with a greater willing to test and use new products, such as EcoProMIS. These enthusiasts are ‘early adopters’. In addition to taking the risk of testing this new product, they also provide feedback on early stage development.

The evaluation of the user experience includes both the users’ perceptions as well as their practice patterns. The feedback process therefore goes beyond simply asking for ‘musical preferences’ to actively inviting users to join the ‘dance’.

 

The Technology Adoption Life Cycle. (www.business-to-you.com)

 

The image above is useful to take the dance metaphor further. It adds the complexity that between the Early Market and the Mainstream Market, there is an open space, called ‘The Chasm’. This is the gap that needs to be traversed before a new product reaches large scale collective adoption.

We saw in the video that as the first three guys are dancing there is a gap before additional members join the group. Once the dancing group crossed the chasm, the new members (the ‘pragmatists’ and ‘conservatives’) arrived en masse.

If a small initial group is willing to dance, to test the product and to provide feedback,  it suggests that the chasm will be crossed and the product can work with other consumers, becoming part of the mainstream market.

Dancing with EcoProMIS

As has been mentioned before on this blog, the EcoProMIS project is working on cutting-edge technology with crop models, an advanced digital platform and data collection with satellites and drones. Like the first dancer in the video, our work is pioneering, bold and at times eccentric!

It is essential for our project to invite others into a ‘dance’. That is why, in parallel to our product development, EcoProMIS is prioritising activities to test our services.

What this means is that throughout the five year project we are inviting Colombian farmers to participate via surveys and meetings. For example, in the coming months we are planning multiple workshops, where each farmer can access the early versions of our mobile apps and knowledge services. These ‘early adopters’, like dancer two and three, will take that initial risk of joining in, while also providing critical early feedback.

This ‘dance’ of feedback and testing will ensure we can better tailor the EcoProMIS platform, apps, and knowledge services. By doing this, we are able to identify the best options and to contribute to our larger goal of improving sustainability and productivity with Colombian oil palm and rice growers.