Engenharia de Dados usando Elastic Stack

23° Meetup Engenharia de Dados – Virtual

Mar. 2021

Ricardo Ferreira

Abstract

O Elastic Stack é capaz de trazer grandes benefícios para adeptos e praticantes de engenharia de dados, ainda que seu uso nem sempre seja óbvio, já que muitos acreditam que a plataforma serve apenas para agregar logs. Esta sessão apresenta uma introdução ao Elastic Stack e mostra como ele pode resolver problemas clássicos de engenharia de dados, abordando integração e reconciliação de dados, técnicas de filtragem, agregação e curadoria, além de mecanismos de consolidação de dados para fins analíticos.

Previous page Building Data-Driven Applications with Elasticsearch

Next page From Zero to Hero with Observability in Microservices

Open source

I contribute to LangChain4j, an idiomatic open source Java library for building LLM-powered applications on the JVM. Recent work includes adding native vector search embedding stores so developers can build RAG, recommendation engines, and AI memory systems.

I also ported RedisVL to Go, an open source, AI-native client that brings vector search, semantic caching, LLM memory, semantic routing, rerankers, and an MCP server to the Redis ecosystem for Golang developers.

Speaking

I’ve been speaking at conferences since 2008 and doing it full time as part of my work with DevRel since 2018. My talks go deep on the systems I build with: distributed systems and event streaming, AI engineering and vector search, and the data infrastructure that has to hold up when the demo ends and production begins. Some of the events I’ve spoken at include AWS re:Invent, Microsoft Ignite, Google Cloud Next, KubeCon, Oracle OpenWorld, QCon, Strange Loop, Kafka Summit, Pulsar Summit, JavaOne, DevNexus, JFokus, JNation, and All Things Open.

Ricardo Ferreira presenting on the main stage at AI DevWorld — On the main stage at AI DevWorld

You can find my upcoming and past talks on my speaking calendar. Recordings also live on my YouTube channel, and the code I write for talks, demos, and tutorials is on my GitHub.

Get in touch

Want to talk about distributed systems, AI engineering, or having me speak at your event? Connect with me through any of the social links below.

Who I am

I work at the intersection of distributed systems, AI, and data infrastructure, turning complex technology into things developers can understand, adopt, and build with.

Lately, that means hands-on AI engineering: building vector search, semantic caching, agent memory, and RAG into the data layer, and figuring out how to make AI agents secure enough to ship. I contribute to open-source projects like LangChain4j and RedisVL for Golang.

The AI-native work isn’t a pivot. It draws on the same systems-design foundation I’ve built for 20+ years: designing data systems for scale, moving data fast, watching where systems break; now applied to vectors and agents. I have worked on RDBMS and Big Data at Oracle; event streaming with Apache Kafka and Apache Flink at Confluent; observability at Elastic; AI and developer tooling at AWS; and NoSQL and vector stores at Redis. That foundation is exactly what separates AI demos that work on stage from AI systems that survive production.

Social Links