2025-02-25, 11:15–11:45, Room 1
Adapting traditional techniques to run efficiently and scale correctly to massive amounts of data.
In this talk, we would like to explore different techniques and strategies we have used in CARTO to provide our users with scalable traditional geospatial techniques, as well as some others that include more bleeding-edge technologies like LLM functionalities. Traditional implementations often run into bottlenecks or untractable complexity when scaling the operations to massive amounts of data. Our challenge is to adapt these methods to rely on cloud-native technologies capable of scaling and to the nuances of each of the computing platforms that they might run on. We will review how to make computation more natural on SQL-like tables, using spatial indexes efficiently, and adapting iterative methods to table-based operations. Some of the geospatial methods to be adapted include creating a spatial composite score, space-time anomaly detection, performing pattern analysis and labeling with LLMs. We have already implemented these functions successfully in Workflows, a product that is part of the CARTO platform and is being used by companies hosting their data in BigQuery, Snowflake, Databricks and other cloud providers.