The rice value chain: better knowledge, best decisions.

The rice value chain: better knowledge, best decisions.

By Nicolás Gonzalez, Business Development Manager at Agricompas

The last few months have been challenging, but also revealing for EcoProMIS and our approach to entering the market. While working hard to develop a comprehensive platform for data analytics, we are doing extensive market research with each of the stakeholders involved in the value chain for rice and agriculture in general.

EcoProMIS is a platform that collects and processes data into analytics from many different sources, providing a complete overview of the production chain. Establishing a link with each of the actors involved in the chain is fundamental. For this reason, our business and product development team work side by side with these stakeholders with the purpose of creating services that generate value and become the source of the company’s revenue.

For the market analysis, which is comprised of the study of both its size and the possible services to offer, a series of interviews and validations have been conducted with each of the stakeholders.

The main stakeholders we are engaging are drawn from the rice milling sector, agri-insurance, credit, seed, machinery, fertilizer and agrochemical suppliers. All of these have a key role in rice production and for which intelligent information is relevant in the decision-making of their business, improving competitiveness and optimising their processes.

Access to information is the critical factor

One of the common factors that the Agricompas team found in the interviews conducted with each of the key actors mentioned above, is that information for them is considered as their guide in daily operations, in the prediction of supply and demand, and in the management of decision making with respect to their business strategy.

Today there is no platform like EcoProMIS and the different actors mentioned above could see clearly the business advantage it would lend them: the possibility of knowing and predicting the yield of farmers’ production, seeing and mitigating risks for the credits that most of them offer, and the management of inventories and supply and demand.

Thanks to this market feedback, we have been able to generate valuable information products, taking our access to satellite images, artificial intelligence, climate information and data provided by farmers themselves, to generate differentiation in the market and value for platform users, making agriculture in Colombia more competitive through better decisions in crop management.

Best services, better conditions for farmers

With the development of the EcoProMIS platform, after market validation and the approval of rice farmers, we are creating a conducive environment for the innovation and growth in the agricultural ecosystem in Colombia.

The information for each of the actors in the chain translates into better products and services for farmers, because with this understanding of production, they are able to better identify their needs and their ability to pay and manage risks, all of which in turn translates into a considerable improvement in the structuring and costing of products.

Technology and access to information are part of the new generation of agriculture that is evolving worldwide. We believe that EcoProMIS is the platform responsible for bringing it to the life of each of the farmers who are part of the project and to the door of each of the companies that are part of the production chain.

EcoProMIS provides users not only with information, but with a valuable tool for daily operations, decision-making, and the way in which users innovate new products and services.

Agricompas and Information Workers working in a big data project for the Agro-Industry

Agricompas and Information Workers working in a big data project for the Agro-Industry

by Carlos Torres, Senior Consultant, IWCO

Information Workers, or IWCO, is an important technical partner working on the EcoProMIS project led by Agricompas.

A Colombian company with 10 years of experience, IWCO is focused on helping its clients get the most and the best value from their data by turning it into information that allows them to make decisions in the short, medium, and long term.

This value comes from four fundamental areas in the process of data harnessing: exploration, extraction, refinement, and consumption of data.

Data Exploration
This phase is designed to understand the types of data, their characteristics, where they are located and their potential value, additionally, the identification of business needs that can be solved with data.

Data Extraction
During the extraction process, we help our clients (such as the EcoProMIS project) to take their data, put it into a structured format that can be simple to use and that enables our clients to immediately create on-demand queries to get answers to their business questions.

Data Refinement
Once the data is in a structured format, we help our clients design and create models to predict, understand, and even extract hidden patterns in the data.

Data Consumption
Business users such as Agricompas and Pixalytics are extremely important, that is why we care about helping them consume their data models – information that can now be prepared for analysis. Through knowledge transfer processes we enable our clients to obtain answers to their strategic questions.

Information Workers is a Microsoft, AWS (Amazon), and Google Partner. With broad experience and a process focused on culture, we have an emphasis on helping organisations to operate in the world of self-service.

We have a multidisciplinary team that includes Mathematicians, Statisticians, Economists, Software Engineers, Systems Engineers, and even Petroleum Engineers (!). This diversity means that our team is able to bring our customers’ different professional perspectives, something that has helped us contribute value in the EcoProMIS consortium of seven international partners.

Together with Agricompas, we created a plan designed to resolve some of the challenges for the EcoProMIS project, namely how we acquire, store, manage, and secure the agricultural analytics data.

After developing this plan, it was essential to choose a platform aligned with these necessities pragmatically, and allow us to obtain results in a short period. Another important step was to identify the sources of big data for the project, for example publicly available data (e.g. earth observation images from UKSA), streaming data from different sensors like weather stations, and data from legacy databases.

Big Data
As we engaged with all of this data, it became obvious that we could not process this multivariate data using traditional methods. The data from the EcoProMIS project is clearly in the realm of big data, complying with the three ‘V’s:

Volume: EcoProMIS has collected a lot of data from different sources.
Velocity: The various data streams are handled within different timeframes from the different sensors.
Variety: All types of formats – from structured to unstructured.

Adding Value
Another key step in our contribution to the project was to find a way to access, manage, and store the data. We needed to find a data platform that can support the storage and the capabilities of analysing petabyte-size files and trillions of objects.

Equally important was prioritising security and data protection, making sure the database is GDPR-compliant, and that data is securely stored. Linked to this is our role in providing data auditing and ongoing support.

Agro-Industry
Big data in the agro-industry plays an important role. With such a large amount of information out there, the data needs to be shaped or tested in a way that adds value to the agro-industry.

By doing so, the agro-industry can better identify problems and reach the goal of sustainable optimisation that is at the heart of the EcoProMIS project.

Connecting growers with an essential service in the modern world

Connecting growers with an essential service in the modern world

By Richard Strange, Head of Engineering at Agricompas

As individuals we all grow in wisdom and capability when we take time to reflect on our actions and find lessons to apply to tomorrow’s challenges. We look at our achievements and the memories that are anchored around them. We use them to guide us in becoming better in both our professional and personal lives. Modern business is very much the same. In the modern world, when a business moves, the byproduct of their actions is data.

Agricultural Data Gap

Whether it is a financial officer’s log of transactions, the record of work hours from an employee’s timesheet, or the number of clicks a website receives each day. It is rare to find a part of a business that isn’t measured or collected, either directly or by proxy through other measures. Yet in agriculture, little information is available around many crucial farming practices that often mean the difference between a bumper crop or financial devastation for families and communities.

It is not enough to say that you have an employee, or a website, or an invoice. The crucial questions are if the employee is doing their work, if the website is drawing attention, if the invoice is correct. Yet farmers are not able to answer critical questions about their own farms. They have sown their seeds, yet cannot say how many are germinating. They apply fertiliser, yet cannot tell if it is cost-effective. By leaving agriculture behind in this wave of data-driven business, the world is abandoning millions of farmers in data poverty, and powerless to compete against their wealthier first-world counterparts.

EcoProMIS Collects Quality Data

In leading the EcoProMIS project, the aim of Agricompas is to make a difference by empowering farmers with the knowledge they need, from sensor to survey to satellite to weather to drone data. But with each additional source of data, the difficulty of pulling them together increases exponentially. I’m the Head of Engineering at Agricompas, and I’m responsible for all the data EcoProMIS gathers. My job is to work out how we pull all this information together, understand it and then provide the information to those that need it.

There are two approaches to tackling a challenge like ours. Firstly, you can manually handle the data, with a team of analysts pushing round files via email, shared folders and collaborative spreadsheets. This does come with the advantage of immediate productivity and visibility. But there’s little certainty over the quality and completeness of data, and no way to be sure what information is where. The second option is to invest time and effort into a fully-fledged platform for data. It must allow the scientists we work with and the farmers that we support to put in and take out the information they need effortlessly.

Advanced Data Platform Prevents Errors

Only recently, the failure of the first, manual approach was highlighted by the loss of the records of 16,000 positive COVID-19 cases by the UK government. Was it a catastrophic server failure? the act of a malicious hacker? The truth was far more mundane. An analyst had opened the spreadsheet holding the list of COVID-19 cases in an old version of Excel, slicing 16,000 rows of data off without ever realising their mistake. Suppose this approach cannot work reliably in the hands of a team as well-staffed as the Public Health England team. How can we trust our own information in a similar system? We owe our growers and our own team better than that.

 

The cost of getting your data platform wrong (https://www.bbc.co.uk/news/uk-54412581)

 

Over the last six months, the EcoProMIS team has been carefully creating a central platform that can look after farmer data responsibly and safely. A system of databases, redundant servers and security measures means that data doesn’t get forgotten, doesn’t get destroyed and doesn’t get leaked. Over the coming months, we are combining our suite of analytics, models and AI with new apps.

These apps will allow farmers to provide and see their data about their farms and help them make the right agricultural decisions. We already have the first app in early tests, with a knowledge presentation app in the works for release by the new year.

Connected Growers

As we evolve our platform and grower apps through close feedback with early users, we will be able to put more power back into the hands of growers, irrespective of their literacy or agricultural experience.

Agricompas and the EcoProMIS project exist to level the playing field and make agriculture fairer for farmers in the most challenging economic, environmental and social settings. I am incredibly proud of what our technical team has achieved to make that happen.

 

 

Data Analytics Make Sense

Data Analytics Make Sense

By Simon Dzurjak, Data Analyst at Agricompas

With the advent of the big data era, the way we gather insights about processes affecting our everyday lives has changed dramatically, and the domain of agriculture and food production is no different.

As a result of the widespread availability of various sensors, from satellites and UAVs, to climate and soil measurement hardware on farms, we can now gather data at unparalleled rates and volumes. However, in its raw state, the data we collect often provides little, if any, insight. These raw measurements need to be processed and analysed through complex pipelines to be turned into a valuable product.

The engine responsible for driving the change of raw data into worthwhile insights is data analytics.

Data Analysis for EcoProMIS

Since joining Agricompas in July, I have been working on ensuring that the data which is stored and arrives on our cloud platform, is as high quality as possible. Analytics and predictions are only ever as good as the data from which they are sourced, hence rigorous checking of data quality is always a necessary layer in data intensive projects. To help with this, I wrote a number of programs generating data quality statistics, which will ensure that any analytics we produce will be reliable and trustworthy.

The statistics produced by these scripts are then visualised and fed into internal dashboards, which provide a quick and intuitive way of keeping an eye on our data and making sure that it looks the way we expect it to.

Data analysis is an essential part of our work on the EcoProMIS project.

 

Eddy Covariance Data

So far, one of my most enjoyable (but challenging) experiences was working with large eddy covariance tower datasets. Eddy covariance towers are brilliant at logging and producing rich atmospheric chemistry flux datasets allowing us to understand the minute gas exchange processes in agricultural fields. This data is important for our crop modellers, allowing them to produce various models capable of predicting crop phenology and yields.

However, due to the way that the hardware transmits the data, it is common for gaps in data to form (e.g. as a result of signal loss). Luckily, with the power of statistics, we can accurately discern the missing values and fill the gaps in the data. Working closely with the crop modelling team, after many discussions and much research, we have successfully automated this process, and allowed ourselves to further iterate and improve on it if need be in the future.

Integrating Diverse Data

As for the future, I also look to my past and hope to bring some of my previous experience in working with remote sensing and Earth observation data into our platform. Integrating and tying all the data types together will be a big challenge, however I am certain that it will bear fruit and truly create a platform which will make farming easier, more precise and more sustainable.

With advances in computational power and the high quality data available to us, I believe that data analytics and artificial intelligence will play a key role in revolutionising agriculture, and it is rewarding to be a part of a team making this happen. Combining multiple sources of data together into a single product will allow us to create a platform capable of supporting decisions via accurate insights and improving the ease with which farmers grow crops like never before.