Ricardo Ferreira

Beyond Prompting: Context Engineering for Production-Grade AI

Mon, 01 Jun 2026 00:00:00 +0000

Dependable AI outputs take more than a well-crafted prompt. In production, you have to coordinate tool calling, memory and retrieval, and token management—all at scale. This session frames context engineering as a discipline: treating context as an architectural resource to be designed, optimized, and managed, not something you improvise per request. We’ll go past theory into implementation—building persistent memory, retrieving relevant information without overwhelming the context window, and managing tokens while preserving coherence. Through a real-world case study, we’ll trace the architectural evolution from a naïve app into a fully engineered context pipeline, with hard-won lessons on improving output consistency and controlling production costs.

In the Land of the AI Agents, the One-Layer Memory Is King

Thu, 07 May 2026 00:00:00 +0000

As AI agents grow more capable, how they remember becomes as important as how they reason. This talk examines memory design for agentic systems, arguing that a well-designed single-layer memory can outperform more elaborate schemes, and explores practical trade-offs in giving agents durable, relevant context without unnecessary complexity.

Building Context-Aware Search Apps with Spring and Redis

Mon, 13 Apr 2026 00:00:00 +0000

This workshop teaches developers how to build applications capable of handling searches that account for user intent, using hybrid search that blends full-text search with vector similarity to improve how users find information. Through hands-on exercises, participants learn to load a dataset into Redis, generate embeddings, model the domain with Spring Data, implement hybrid queries, and monitor Redis for optimizations, connecting these concepts to the recommendation systems behind services like Amazon and Netflix.

Beyond Prompting: Context Engineering with LangChain4J

Wed, 08 Apr 2026 00:00:00 +0000

Reliable LLM outputs require far more than good prompting: developers must handle tool calling, memory, retrieval pipelines, and structured outputs while keeping consistency across many interactions. This session introduces context engineering as a discipline that treats the model’s context window as an architectural resource to design and manage, and shows through live coding how LangChain4J implements it, building an agentic system with tool calling, short- and long-term memory, retrieval-augmented generation with vector search, and type-safe structured outputs, contrasting naive prompting with an engineered approach to demonstrate gains in accuracy, consistency, and token efficiency.

The Seven Deadly Sins with MCP

Fri, 03 Apr 2026 00:00:00 +0000

MCP is quickly becoming the standard for connecting LLMs to real systems, and with that reach comes a whole new set of ways to get it wrong. This session walks through seven of them, mapped to the seven deadly sins: gluttony (resource abuse), sloth (lazy error handling), wrath (aggressive polling), greed (permission overreach), pride (overengineering), envy (tool sprawl), and lust (unsafe exposure). Using a scenario where an MCP server is handed to an overzealous LLM—triggering memory leaks, runaway polling, and dangerously broad permission scopes—we’ll look at the patterns and symptoms behind each sin, and the practices that keep MCP deployments dependable. The goal: ship with absolution, instead of yet another pager alert.

Context Engineering with Redis and LangChain

Tue, 31 Mar 2026 00:00:00 +0000

This session teaches how to structure memory, retrieval, and reasoning workflows so that language models perform dependably in production. It shows how Redis and LangChain together enable high-performance agent systems through semantic caching, vector search, and intelligent context management.

Getting Started with the MCP Server for Redis Cloud using Docker

Thu, 19 Mar 2026 00:00:00 +0000

A getting-started guide to running the Model Context Protocol (MCP) server for Redis Cloud in Docker, so AI assistants can work directly with your Redis data.

Beyond Prompting: Context Engineering with LangChain4J

Fri, 20 Feb 2026 00:00:00 +0000

Speed Up your Apps with RDI and Redis Cloud

Mon, 01 Dec 2025 00:00:00 +0000

Applications often slow down because they depend on primary databases that were never designed for high-throughput, low-latency access. This talk shows how Redis Data Integration (RDI) can continuously sync data from existing databases into Redis Cloud in near real time, letting you place a fast, always-fresh caching and data layer in front of your systems to dramatically speed up application performance without rewriting your source of truth.

To Vector, or not to Vector, that is the Question

Mon, 24 Nov 2025 00:00:00 +0000

This year, I presented a session at the All Things Open conference called To Vector, or not to Vector, that is the Question, and I found myself facing a packed room with people standing in the back. The topic? Whether you actually need a vector database. It seems I wasn’t the only one struggling with this question. This session was very well received, and the most unanimous feedback I received after the session was whether a blog about this session would be written or not.

To Vector, or not to Vector, that is the Question

Mon, 13 Oct 2025 00:00:00 +0000

A data-driven framework for making smart decisions about vector databases. This session explores when you actually need a vector database and when a simpler solution wins, covering the hidden costs of embeddings, infrastructure, and debugging.

An Alexa Skill that Recalls All Memories

Tue, 30 Sep 2025 00:00:00 +0000

A short demo of an Alexa skill that stores and recalls memories on demand, showing how voice interfaces can front a simple memory store.

Vector Sync Patterns: Keeping AI Features Fresh When Your Data Changes

Thu, 04 Sep 2025 00:00:00 +0000

The common advice to just rebuild your vectors nightly breaks down when source data and application code change constantly, leaving AI features stale, driving up compute costs, and frustrating users. This session explores real-world synchronization patterns that work across different database ecosystems, covering event-driven architectures that update vectors in real time, smart change detection that reprocesses only what is necessary, and resilient mapping layers that insulate applications from repeated refactoring as data models evolve.

Reimagining your AI Workloads with Redis

Wed, 13 Aug 2025 00:00:00 +0000

A look at how Redis can accelerate and simplify modern AI workloads. The session covers using Redis as a vector database for semantic search and retrieval-augmented generation, along with caching, feature storage, and low-latency data access patterns that help teams build responsive, production-grade AI applications.

Vector Database Search: HNSW Algorithm Explained

Mon, 28 Jul 2025 00:00:00 +0000

A walkthrough of how the Hierarchical Navigable Small World (HNSW) algorithm powers fast approximate nearest-neighbor search in vector databases, and why it strikes such a good balance between speed and accuracy.

Is Your Vector Database Really Fast?

Mon, 21 Jul 2025 00:00:00 +0000

Not all vector databases are as fast as their benchmarks claim. This video digs into what actually determines vector search performance and how to measure it honestly for your own workload.

What is Filtered Search?

Tue, 15 Jul 2025 00:00:00 +0000

A short introduction to filtered search: how to combine metadata filters with vector similarity so you retrieve results that are both semantically relevant and constrained to the right subset of your data.

What is Hybrid Search?

Fri, 04 Jul 2025 00:00:00 +0000

An explanation of hybrid search, which blends traditional keyword (lexical) matching with vector similarity to get the best of both worlds when building retrieval systems.

Chunking Strategies Explained

Fri, 27 Jun 2025 00:00:00 +0000

How you split your documents into chunks has a big impact on retrieval quality. This video breaks down the common chunking strategies and when to reach for each one.

Vector Sync Patterns: Keeping AI Features Fresh When Your Data Changes

Mon, 09 Jun 2025 00:00:00 +0000

Building Context-Aware Applications with Java, Vector Searches, and Redis

Wed, 28 May 2025 00:00:00 +0000

Context-aware applications, including those powered by large language models, rely on retrieving semantically relevant information rather than exact keyword matches. This session shows how to build such applications in Java by generating vector embeddings and using Redis as a vector database to perform similarity searches, enabling features like semantic search and retrieval-augmented generation.

From Data to Actionable Insights: Building a Bluesky bot powered by Spring AI

Thu, 22 May 2025 00:00:00 +0000

Turning raw data into meaningful action is where modern AI applications earn their value. This session walks through building a Bluesky bot with Spring AI, showing how to combine data ingestion, large language models, and Spring’s AI abstractions to transform a continuous stream of social data into automated, actionable insights.

Long-Term Memory with LangGraph

Mon, 12 May 2025 00:00:00 +0000

A deep look at implementing long-term memory in agentic applications with LangGraph, so your agents can retain and recall information across sessions.

Short-Term Memory with LangGraph

Mon, 05 May 2025 00:00:00 +0000

How to give your LangGraph agents short-term memory, letting them keep track of context within a conversation or task before it is persisted or discarded.

Create a New Database on Redis Cloud

Sat, 08 Mar 2025 00:00:00 +0000

A step-by-step walkthrough of creating a new database on Redis Cloud to back your applications.

Similarity Search using Vector Store

Fri, 07 Mar 2025 00:00:00 +0000

Implementing similarity search over embeddings using a vector store to power retrieval in AI apps.

A Semantic Cache using LangChain

Thu, 06 Mar 2025 00:00:00 +0000

Building a semantic cache with LangChain and Redis to avoid redundant LLM calls for similar queries.

LLM Session Management with Redis

Wed, 05 Mar 2025 00:00:00 +0000

How to manage LLM conversation sessions using Redis to persist and retrieve chat state.

Resources to Learn AI with Redis

Tue, 04 Mar 2025 00:00:00 +0000

A roundup of resources to help you get started learning AI development with Redis.

Building AI Apps using LangChain

Mon, 03 Mar 2025 00:00:00 +0000

A short intro to a series on building AI applications with LangChain and Redis.

The Engineering Wisdom Behind Redis's Single-Threaded Design

Thu, 27 Feb 2025 00:00:00 +0000

In the relentless pursuit of performance, our industry often gravitates toward seemingly obvious solutions: more cores, more threads, more concurrency. Yet Redis—one of the most performant databases in the world—has maintained its commitment to a primarily single-threaded execution model. As various Redis forks emerge claiming dramatic performance improvements through multi-threading, I want to explore why Redis’s core architectural choices remain fundamentally sound, even at scale.

Counterintuitive Brilliance of Single-Threaded Design

When Salvatore Sanfilippo designed Redis, he made a choice that seemed to contradict conventional wisdom: a single-threaded event loop at the core of a high-performance database. This wasn’t a limitation to be overcome—it was a deliberate architectural decision that eliminated entire classes of complexity:

Redis Insight: A Developer's Deep Dive

Tue, 18 Feb 2025 00:00:00 +0000

A hands-on deep dive into Redis Insight, the GUI for exploring, debugging, and optimizing your Redis data from a developer’s perspective.

What AI Coding Assistants Have in Common with iRobot Roomba?

Sun, 05 Jan 2025 00:00:00 +0000

In 2010, I purchased my first iRobot Roomba, the 700 series model launched that year. It promised to revolutionize automated cleaning, a claim repeatedly made since the device’s initial release in 2002. Unaware of previous models’ shortcomings, I was captivated by the promises of this new version. I recall reading about patented features like the Dirt Detect Technology™ and built-in integration with voice assistants. All of which seemed incredibly innovative at the time.

Getting Started with the VSCode Extension for Redis

Mon, 16 Dec 2024 00:00:00 +0000

A getting-started guide to the Redis extension for Visual Studio Code, so you can browse keys and run commands against Redis without leaving your editor.

Getting Started with Amazon Q Developer Customizations

Wed, 17 Jul 2024 00:00:00 +0000

An introduction to Amazon Q Developer customizations, which let you tailor code suggestions to your organization’s own codebase and conventions.

Sending Traces to AWS X-Ray Using Amazon Q Developer

Thu, 16 May 2024 00:00:00 +0000

A practical example of using Amazon Q Developer to instrument an application and send distributed traces to AWS X-Ray.

Implementing an OpenTelemetry Collector Using Amazon Q Developer

Wed, 15 May 2024 00:00:00 +0000

Walking through how to stand up an OpenTelemetry Collector with the help of Amazon Q Developer, from configuration to running pipelines.

Using Amazon Q Developer to Write Code for Observability

Tue, 14 May 2024 00:00:00 +0000

How Amazon Q Developer can accelerate writing observability code, generating the instrumentation and boilerplate so you can focus on what to measure.

Supercharging Alexa with Amazon Bedrock Foundation Models

Tue, 28 Nov 2023 00:00:00 +0000

Combining a familiar voice assistant with generative AI opens up richer, more conversational experiences. This session shows how to supercharge Alexa skills using foundation models from Amazon Bedrock, connecting voice interactions to large language models so that responses become more natural, context-aware, and capable of handling open-ended questions beyond scripted intents.

Deep Dive into the Queue Data Structure

Thu, 19 Oct 2023 00:00:00 +0000

A detailed exploration of queues: FIFO semantics, common implementations, and where they fit in real-world systems.

Developing Java Applications using Amazon Bedrock

Tue, 17 Oct 2023 00:00:00 +0000

A guide to building Java applications on top of Amazon Bedrock, connecting your code to foundation models through the AWS SDK.

Deep Dive into the Linked List Data Structure

Thu, 05 Oct 2023 00:00:00 +0000

A deep dive into linked lists: singly and doubly linked variants, their trade-offs against arrays, and the operations that make them shine.

Deep Dive into the Array Data Structure

Fri, 29 Sep 2023 00:00:00 +0000

A thorough deep dive into arrays: how they are laid out in memory, their performance characteristics, and the operations they support.

Deep Dive into the Matrix Data Structure

Fri, 29 Sep 2023 00:00:00 +0000

An in-depth look at matrices as a data structure, covering representation, traversal, and the common operations you will encounter.

Learning Data Structures with Amazon CodeWhisperer

Fri, 29 Sep 2023 00:00:00 +0000

Using Amazon CodeWhisperer as a pair-programming companion to learn and implement classic data structures from scratch.

Testing Terraform Providers using TestContainers for Go

Tue, 25 Jul 2023 00:00:00 +0000

Writing reliable functional tests for Terraform providers is difficult when they depend on real backing services. This livestream shows how to use Testcontainers for Go to spin up those dependencies in disposable Docker containers, so provider acceptance tests run consistently and reproducibly against real infrastructure.

AWS Databases: Use Cases and Optimizations

Thu, 13 Jul 2023 00:00:00 +0000

An overview of the AWS database portfolio, matching common use cases to the right service and sharing optimization tips along the way.

A Three-Minute Summary of What to Consider When Choosing Databases

Tue, 20 Jun 2023 00:00:00 +0000

A quick, three-minute summary of the key trade-offs to weigh when choosing a database for your workload.

The Right Number of Partitions for a Kafka Topic

Wed, 05 Apr 2023 00:00:00 +0000

Every technology has that key concept people struggle with, and for Apache Kafka the winner is how many partitions to set for a topic. Sizing partitions wrongly affects storage, parallelism, durability, and how much load Kafka can handle, and while it is often treated as an infrastructure decision left to Ops teams, it is really an architectural design decision that even affects how much code you write. This session peels off the concept of partitions from the perspective of the cluster and its clients, explains the formula to decide how many partitions a topic should have, and shows how to spot a poor decision when you see one.

The Right Number of Partitions for a Kafka Topic

Tue, 04 Apr 2023 00:00:00 +0000

Every technology has that key concept that people struggle to understand. With databases, is which join clause to use for fetching data from multiple tables. Containers are tricky when you have to pick a storage type given some persistence requirements. With Apache Kafka, the winner is how many partitions to set for a topic. Why this is important? You may ask. Well, sizing Kafka partitions wrongly affects many aspects of the system, such as storage, parallelism, and durability. Worse, it may also affect how much load Kafka can handle. Hence why often the decision about how many partitions to set for a topic is handled by Ops teams, as we see this to be only an infrastructure matter. In reality, this is an architectural design decision that affects even the amount of code you write. This session will peel off the concept of partitions and explain it from the perspective of the Kafka cluster and its clients. It will explain the formula people should use to decide how many partitions to set for a topic, and how to spot a poor decision when they see one.

BOA301: Building your own Apache Kafka connectors

Mon, 28 Nov 2022 00:00:00 +0000

Apache Kafka Connect provides a framework for streaming data between Kafka and external systems, and while many connectors exist, teams often need to build their own for custom sources and sinks. This hands-on workshop teaches how to develop a custom Apache Kafka connector and deploy it on AWS, following the right best practices for building reusable, production-ready connectors.

Changing the Backend to AWS with No Code Changes

Mon, 14 Nov 2022 00:00:00 +0000

Switching your telemetry backend to AWS without touching application code, thanks to the Collector.

Sending the Generated Metrics to Prometheus and Grafana

Sun, 13 Nov 2022 00:00:00 +0000

Exporting OpenTelemetry metrics to Prometheus and visualizing them in Grafana dashboards.

Creating Custom Metrics with the OpenTelemetry SDK for Java

Sat, 12 Nov 2022 00:00:00 +0000

How to define and record custom metrics in Java applications using the OpenTelemetry SDK.

Sending Generated Traces to Grafana Tempo

Fri, 11 Nov 2022 00:00:00 +0000

Configuring your pipeline to send OpenTelemetry traces to Grafana Tempo for storage and querying.

Getting Started with the OpenTelemetry Collector

Thu, 10 Nov 2022 00:00:00 +0000

An introduction to the OpenTelemetry Collector: receiving, processing, and exporting telemetry data.

Automatic Instrumentation with the OpenTelemetry Agent

Wed, 09 Nov 2022 00:00:00 +0000

Using the OpenTelemetry Java agent to automatically instrument your application with zero code changes.

Creating Custom Traces with the OpenTelemetry SDK for Java

Tue, 08 Nov 2022 00:00:00 +0000

How to create custom spans and traces in Java using the OpenTelemetry SDK for fine-grained visibility.

Instrumenting Java Applications using OpenTelemetry

Mon, 07 Nov 2022 00:00:00 +0000

The start of a series on instrumenting Java applications with OpenTelemetry to capture traces and metrics.

When Kafka is the source of truth; schemas become your source of headaches

Wed, 05 Oct 2022 00:00:00 +0000

An exploration of the challenges that arise when Apache Kafka becomes the system of record and schemas must evolve over time. The session covers schema management with a schema registry, compatibility strategies, and serialization formats such as Avro, Protobuf, and JSON Schema, offering practical approaches to evolve data contracts without breaking downstream consumers.

When Kafka is the source of truth; schemas become your source of headaches

Tue, 04 Oct 2022 00:00:00 +0000

One of the coolest things about streaming systems such as Apache Kafka is their ability to handle any type of data. You can store events at Kafka and have different systems processing their event data. You may start with a few systems and add new systems as needed. While certainly possible and attractive, this isn’t simple. Schemas play a key role in how each system consumes the event data and processes them. Reason Schema Registry exists, right? Not really. Schema Registry doesn’t solve any of your data problems. It’s just a registry for your schemas. Admittedly, without it, there would be no policy enforcement. However, data problems can still happen. Issues with encoding, format mismatch between different programming languages, new code not being able to read data written by old code, etc. In this session, we will get into the weeds of data serialization with schemas. We will discuss the differences between formats like JSON, Avro, Thrift, and Protocol Buffers, and how your code must use each one of them to serialize data. It will also clarify the impact of switching Schema Registry with other registries, and whether you can use them together. If you ever wondered why your Python code can’t read something written by Java, why integers are getting confused with strings, or simply how schemas end up in Schema Registry, this session is for you.

Working as event host along with Jacquie Grindrod

Thu, 29 Sep 2022 00:00:00 +0000

Ricardo Ferreira co-hosted Build On Observability Day alongside Jacquie Grindrod, guiding the live stream by introducing speakers and sessions, framing the day’s themes around observability, and moderating discussion between segments. Rather than delivering a solo talk, the role focused on connecting the audience with the presenters and keeping the observability-focused event flowing.

In the land of the sizing, the one-partition Kafka topic is king

Sat, 24 Sep 2022 00:00:00 +0000

This talk explores one of Apache Kafka’s most challenging concepts: determining the right number of partitions for a topic. Using a “what-if” presentation style, it examines how partition sizing affects system qualities such as consistency, concurrency, durability, and overall load capacity, highlighting the consequences of poor sizing decisions and whether recovery from them is possible.

In the land of the sizing, the one-partition Kafka topic is king

Wed, 21 Sep 2022 00:00:00 +0000

Every technology has that key concept that people struggle to understand. With databases, is which join clause to use for fetching data from multiple tables. Containers are tricky when you have to pick a storage type given some persistence requirements. With Apache Kafka, the winner is how many partitions to set for a topic. Why this is important? You may ask. Well, sizing Kafka partitions wrongly affects many aspects of the system, such as consistency, concurrency, and durability. Worse, it may also affect how much load Kafka can handle. Hence why often the decision about how many partitions to set for a topic is handled by Ops teams, as we see this to be only an infrastructure matter. In reality, this is an architectural design decision that affects even the amount of code you write. This session will peel off the concept of partitions and explain it from the perspective of the Kafka cluster and its clients. By using a what-if presentation style, it will explain the overall impact on the system given a number. This will help you build more confidence about how to size Kafka partitions correctly, and to spot a poor decision when you see one.

Is Using KoP (Kafka-On-Pulsar) a Good Idea?

Thu, 18 Aug 2022 00:00:00 +0000

Building microservices around Apache Kafka is your job, and life is great. One day, you hear community members talking about some neat Apache Pulsar features, and get you intrigued. I mean, we all love Kafka, but you can’t avoid wondering if migrate one of your projects to Pulsar is a good idea. Then it happens. You find Pulsar supports Kafka clients natively via a protocol handler called KoP: Kafka-On-Pulsar. This gets you pumped. Is that it? Can I go ahead and simply point my microservices to Pulsar and be a hero with this migration? But you must be responsible; and history says you shouldn’t believe migrations like this are refactoring free. Reason you may get interested in this session. We will revisit the architecture behind protocol handlers to understand what it means having one enabled on Pulsar. Plus, we will discuss the internals of KoP. Finally, we will use a show-and-tell approach to detail the effort to migrate a microservice written for Kafka to Pulsar, and whether the code need to change for this.

No approved talks. Joining the conference as an attendee

Tue, 19 Jul 2022 00:00:00 +0000

Do It Yourself: Programmable Metrics using OpenTelemetry

Tue, 14 Jun 2022 00:00:00 +0000

Using metrics to measure how good or bad things are going is a proven way to ensure a software-based system is going in the right direction. Most metrics are created and monitored automatically by agent technologies installed in our infrastructure, making us hostages of the set of metrics that these agents are programmed to address. But what if you need to handle your own set of metrics? This is a question that often drives developers mad because they fear spending development cycles building something that will end up being locked into a particular monitoring/observability vendor. But OpenTelemetry — a CNCF observability framework that provides a vendor-neutral approach to tackle metrics, logging, and tracing needs, can change everything. This talk will explain how the OpenTelemetry framework allows the creation of custom metrics in a standard, scalable, and reusable way. It will provide an example in Java of a set of metrics that are continuously updated based on the execution of the code and how to hook that data with a compatible observability backend.

Distributed Tracing: The good, The Bad, and the Ugly

Tue, 24 May 2022 00:00:00 +0000

Let’s face it: most people talking about observability all end up talking about distributed tracing somehow. It is a technology that radically changes how we identify and solve technical problems. In a world where virtually all applications are born distributed — it seems to be something that you ought to know in more detail. This talk will provide a pragmatic overview of distributed tracing by clearly articulating its motivation, problems it solves, the challenges, and what technologies you should use to ensure a vendor-agnostic implementation. It will discuss things like white-box and black-box instrumentation, the pros and cons of each approach, and why most developers prefer one versus the other.

Building Software Reliability with Distributed Tracing

Tue, 10 May 2022 00:00:00 +0000

As applications grow into distributed systems of microservices, understanding how a single request travels across many components becomes essential to keeping them reliable. This session explains how distributed tracing works, how it complements metrics and logs as a pillar of observability, and how instrumenting services lets teams pinpoint latency, errors, and bottlenecks to build more reliable software.

Down to the Rabbit Role with the OpenTelemetry Agent

Thu, 28 Apr 2022 00:00:00 +0000

Instrumenting code by hand is not always practical, which is where the OpenTelemetry agent and auto-instrumentation come in. This session follows the rabbit hole of how the agent attaches to running applications to capture traces and metrics with little or no code changes, what it can and cannot see automatically, and how SREs can use it to gain observability into services they do not own.

From Zero to Hero with OpenTelemetry

Wed, 27 Apr 2022 00:00:00 +0000

OpenTelemetry has become the vendor-neutral standard for instrumenting applications with traces, metrics, and logs. This talk takes developers from the fundamentals to a working setup, covering the core concepts, the SDKs and collector, and how to add meaningful observability to real applications without lock-in.

How to Create more Effective Developer Content

Thu, 24 Mar 2022 00:00:00 +0000

Let’s say you create developer content for a living. In that case, chances are that you work in one of those fancy jobs titles like Developer Advocacy, Community Management, Product Technical Marketing, Program Management, and many others. Regardless of your title, what is essential is for you to apply effectiveness to your content, which means that you do want that your content targets the right audience and be as relevant as possible. This blog post will discuss how you can accomplish this by using a technique that I affectionately call the 🔺 triangle. This technique is about making your content focus on the different ways people learn, creating the appropriate relevancy by linking them together, and helping scale the content promotion.

Building Software Reliability with Distributed Tracing

Thu, 03 Mar 2022 00:00:00 +0000

Elastic and AWS Tech Talk + Artisanal coffee

Wed, 26 Jan 2022 00:00:00 +0000

An informal tech talk session pairing a deep dive into observability with the Elastic Stack on AWS with a relaxed, coffee-in-hand atmosphere. The session covers how to build end-to-end observability—logs, metrics, and traces—for applications running on AWS using Elastic Observability, with practical guidance for getting started.

OpenTelemetry for Dummies: Instrumenting Go Apps

Tue, 14 Dec 2021 00:00:00 +0000

Observability starts with good instrumentation, and OpenTelemetry has become the vendor-neutral standard for producing traces, metrics, and logs. This beginner-friendly session shows Go developers how to instrument their applications with the OpenTelemetry SDK, propagate context across services, and export telemetry to the backend of their choice, without locking themselves into any single vendor.

Building Connectors for Apache Kafka with Java

Thu, 09 Dec 2021 00:00:00 +0000

A developer-focused session on extending Apache Kafka with custom connectors written in Java. The talk covers the Kafka Connect framework, the source and sink connector APIs, configuration and offset management, and error handling, giving attendees the knowledge to integrate external systems with Kafka in a scalable and reliable way.

Misturando Código de Aplicação com Infraestrutura no Terraform

Thu, 02 Dec 2021 00:00:00 +0000

Terraform costuma ser usado para provisionar infraestrutura, mas cada vez mais equipes precisam combinar código de aplicação e infraestrutura em um mesmo fluxo de trabalho. Esta sessão explora como misturar lógica de aplicação com a definição de infraestrutura no Terraform, mostrando padrões, provedores e técnicas para orquestrar recursos e comportamento de aplicação de forma coesa e reproduzível.

Observability for your AWS Environment, as you Modernize

Mon, 29 Nov 2021 00:00:00 +0000

As organizations modernize their workloads on AWS with microservices, containers, and serverless, monitoring alone is no longer enough to understand system behavior. This session covers how to build observability into a modern AWS environment, using traces, metrics, and logs to gain end-to-end visibility across distributed and cloud-native architectures.

The Subtle Art of Giving Code Walkthrough Talks

Wed, 10 Nov 2021 00:00:00 +0000

Live code walkthroughs are among the most engaging conference formats, but also the easiest to get wrong. This talk shares practical techniques for structuring and delivering code-centric presentations that keep an audience oriented, avoid common live-coding pitfalls, and turn source code into a compelling narrative.

The Subtle Art of Giving Code Walkthrough Talks

Mon, 08 Nov 2021 00:00:00 +0000

If your audience is developers, then there is a high chance that you will need to show how code works in one of your talks. While you can always play safe and show snippets of code through slides — giving people code walkthroughs is often a more effective method to see the code where it really matters: in the development environment. But doing so can be a daunting task. Not so much because you may not know the code — but because now your content is the code. So many moving parts now need to be addressed by your code walkthrough, the talk’s timing, how much the audience will absorb, the outcomes you want people to have, and how exciting your talk will be if using this method. In this talk, Ricardo will explain what best practices you should follow to nail your following code walkthrough, so your talk becomes memorable. It will do it by providing an utterly wrong code walkthrough example that will be polished during the talk to highlight each best practice.

What is Distributed Tracing and, Why Should you Care About it?

Tue, 19 Oct 2021 00:00:00 +0000

Any discussion about observability inevitably turns to distributed tracing, a technology that is radically changing the way we identify and solve technical problems. Because nearly all modern applications are inherently distributed, this session gives a practical overview of the reasons behind distributed tracing, the problems it solves, its challenges, and the tools that enable a vendor-agnostic implementation, including how to choose an observability backend and the distinction between white-box and black-box instrumentation.

What is Distributed Tracing, and Why Should you Care About it?

Mon, 18 Oct 2021 00:00:00 +0000

Let’s face it: most people talking about o11y (observability) all end up talking about distributed tracing somehow. It is a technology that is radically changing the way we identify and solve technical problems. In a world where virtually all applications are born distributed — it seems to be something that you as an SRE ought to know in more detail. This talk will provide a pragmatic overview of distributed tracing by clearly articulating its motivation, problems it solves, the challenges, technologies you should use to ensure a vendor-agnostic implementation, and which aspects you should consider while picking an o11y backend. While discussing the challenges, this talk will highlight white-box versus black-box instrumentation, which is valuable knowledge to determine where the developer’s responsibility finishes and when the Ops team starts, and — when both team’s responsibilities may entangle.

Take me Down to the Paradise City Where the Metric is Green and Traces are Pretty

Tue, 12 Oct 2021 00:00:00 +0000

Observability is a software discipline that goes back to when virtually any problem could be solved by tailing web server logs. But the world has changed, and systems these days are comprised of different services running in their own stacks who cooperatively build up what we understand as the end-to-end architecture. Thus, observability had to evolve as well. Today we have OpenTelemetry—an observability framework for cloud-native software. OpenTelemetry provides the tools, APIs, and SDKs to create a reusable, robust, and non-vendor-driven observability strategy. But the reality is that most developers are still confused about the lines that separate OpenTelemetry from the past and which parts of the framework are stable enough to be used in production. This talk will explain how OpenTelemetry works and provide examples in Java and Go to illustrate the APIs you can use to produce traces and metrics.

Building Observable Streaming Systems with Elastic APM

Wed, 06 Oct 2021 00:00:00 +0000

Event streaming systems built on technologies like Apache Kafka are inherently distributed and asynchronous, which makes understanding their behavior in production a real challenge. This session shows how to instrument streaming applications with Elastic APM and distributed tracing so that events can be followed as they flow across producers, brokers, and consumers, giving teams the visibility they need to diagnose latency and failures in real time.

Take me Down to the Paradise City where Data Streams are Easy with Storage to Infinity

Wed, 06 Oct 2021 00:00:00 +0000

Data streaming platforms have historically been constrained by how much data they can retain, forcing hard choices between cost and history. This session explores tiered and infinite storage for streaming data, showing how decoupling compute from durable object storage lets teams keep streams around indefinitely and replay them whenever they are needed.

Writing Custom Sink Connectors for Pulsar I/O

Wed, 06 Oct 2021 00:00:00 +0000

You currently have massive amounts of data sitting on Apache Pulsar and likely — even more data coming in every second. But part of your project requires this data to be sent to some system to explore the dataset further. You look into the Pulsar website for built-in connectors to that particular system and realize that there are none. What a bummer. But you don’t have to give up on Pulsar because of this. You can write custom sink connectors for Pulsar I/O that work just like the built-in ones, and all you will need is a bit of Java development experience and creativity. The rest you can leave to this talk. This talk will explain how to start developing your custom connector, show how the code looks like, demonstrate how to deploy, and discuss some of the design decisions that your custom connector may need to address.

Boosting your Tests with Elasticsearch using TestContainers-Go

Thu, 30 Sep 2021 00:00:00 +0000

TestContainers is a popular framework that allows Java developers to create tests that depend on backend systems such as Elasticsearch. It automatically creates a container instance for the said backend, preventing developers from manually spinning up their dependency instances. Also, it enables them to treat those instances as disposable, meaning that once the tests finish executing, the underlying containers created for the instances are destroyed automatically.

The good news for Go developers is that there is a version of this for them as well. It is called TestContainers-Go. In this post, I will walk you through creating a simple test that connects with Elasticsearch using this framework.

Work Smarter; Not Harder: Using ML to Observe your Kuma API Metrics

Tue, 28 Sep 2021 00:00:00 +0000

Service meshes like Kuma expose a wealth of metrics about API traffic, but manually watching dashboards and setting static thresholds does not scale as systems grow. This talk shows how machine learning can be applied to the metrics emitted by Kuma to automatically detect anomalies and surface meaningful signals, helping teams observe the health of their APIs more intelligently and react before issues impact users.

Elasticsearch para Gophers

Wed, 25 Aug 2021 00:00:00 +0000

Uma introdução prática ao uso do Elasticsearch em aplicações escritas em Go. A palestra mostra como conectar programas Go ao Elasticsearch, indexar e consultar documentos e tirar proveito de busca full-text e agregações, apresentando padrões idiomáticos e boas práticas para desenvolvedores Gophers que precisam de busca e analytics em tempo real.

Streaming de Dados usando Elasticsearch

Fri, 20 Aug 2021 00:00:00 +0000

Além de busca e análise, o Elasticsearch pode ser peça central em arquiteturas de streaming de dados em tempo real. Esta sessão apresenta como construir pipelines que ingerem, enriquecem e indexam fluxos contínuos de eventos no Elasticsearch, discutindo padrões de integração e boas práticas para analisar dados no momento em que são gerados.

Elasticsearch for Gophers

Tue, 03 Aug 2021 00:00:00 +0000

Join us as Elastic Developer Advocate Ricardo Ferreira demos how to index, query, search, and aggregate data using the Elasticsearch Client for Go.

Ultimate Guide to the Terraform Provider for Elastic

Fri, 16 Jul 2021 00:00:00 +0000

Most developers deploy their Elastic (Elasticsearch and Kibana) clusters manually by copying the bits to the target system or semi-manually using containers. This might work well if you need to maintain few clusters for your company, where likely you know the name of each one from the top of your head. However, this approach doesn’t scale very well if you have to spin up clusters on-demand for seasonal requirements.

Elastic solves this problem by providing Elastic Cloud — a family of products that allows you to treat Elastic clusters as cloud resources and manage them as if they were one. There are three options:

How to Check SSL Certificate Expiration

Mon, 28 Jun 2021 00:00:00 +0000

Using Heartbeat to monitor SSL certificate expiration so you are alerted before certificates lapse.

Adding Geo-Information to Heartbeat Monitors

Sun, 27 Jun 2021 00:00:00 +0000

Enriching Heartbeat monitors with geo-information so you can see availability from a geographic perspective.

Heartbeat Command-Line Options

Sat, 26 Jun 2021 00:00:00 +0000

A look at Heartbeat’s command-line options and how to use them to control its behavior.

How to Download, Install & Configure Heartbeat

Fri, 25 Jun 2021 00:00:00 +0000

A getting-started guide to downloading, installing, and configuring Heartbeat for uptime monitoring.

Monitoring Docker Containers with Autodiscover

Thu, 24 Jun 2021 00:00:00 +0000

How to use Heartbeat’s autodiscover feature to automatically monitor Docker containers as they come and go.

Everything you Always Wanted to Know about Heartbeat

Wed, 23 Jun 2021 00:00:00 +0000

The opening video of a series on Heartbeat, Elastic’s uptime and availability monitoring shipper.

Down to the Rabbit Hole with Pulsar I/O

Wed, 16 Jun 2021 00:00:00 +0000

Apache Pulsar is a distributed messaging and streaming platform that stores messages durably and scalably into its persistent store, making it an attractive technology to store business data. However, merely storing the data is not enough. To make this data useful, the platform must also provide ways to ingest new data and send existing data elsewhere. While developers can build applications for this using the client libraries, the reality is that most of them don’t want to spend time writing code for repeatable tasks such as — reading data from a database and storing it into Pulsar. Reason why Pulsar abstracts away things like this by providing a connector-based framework called Pulsar I/O. This talk will provide an overview of how the Pulsar I/O framework works and a deep dive into troubleshooting things — from identifying when the connector is not working correctly to more elaborating investigations that may be useful for debugging purposes. It will give you the required tools to master how to ingest and export data into and out of Pulsar effectively.

Hidden Secrets about Instrumenting JVMs for OpenTelemetry

Wed, 16 Jun 2021 00:00:00 +0000

OpenTelemetry is the new-kid-on-the-block in API to instrument distributed applications to send telemetry data (logs, metrics, and traces) to backends for visualization and monitoring. It was created to be a de-facto replacement to standards such as OpenTracing and OpenCensus. The support for Java in OpenTelemetry is generous. It supports both automatic and manual instrumentation. With automatic instrumentation, developers can have known libraries and frameworks instrumented without writing a single code line. Manual support also allows them to decide which parts of the code need instrumentation, and there is a rich set of APIs available to use. This talk will open the pandora box for both options. It will reveal which knobs are available to use that make a tremendous difference in how the applications report telemetry data — both in terms of performance and details. Knowing these hidden secrets will grant developers fine control over their applications and how observability stacks show them to the world.

Building Observable Streaming Systems with OpenTelemetry

Tue, 15 Jun 2021 00:00:00 +0000

Streaming systems built on technologies like Apache Kafka are inherently distributed and asynchronous, which makes understanding their behavior a challenge. This session shows how to make streaming systems observable using OpenTelemetry, propagating context across producers and consumers to capture distributed traces and metrics that reveal end-to-end data flow and latency.

Building Observable Streaming Systems with OpenTelemetry

Mon, 14 Jun 2021 00:00:00 +0000

Building streaming systems is a popular way for developers to implement applications that react to data changes and process events as they happen. It is an exciting new world that technologies like Apache Pulsar made available for anyone to use. But all this goodness doesn’t come for free. One of the challenges of this type of architecture is that its distributed nature makes it hard and sometimes even impossible to identify the root cause of problems quickly. That is why distributed tracing technologies are so important. By gluing together disparate services into a single and cohesive transaction, developers can provide to the operations team a way to pragmatically observe the system and to quickly identify the root cause of problems such as slowness and unavailability. This talk will explain how to implement distributed tracing in Pulsar applications using OpenTelemetry—an observability framework for cloud-native software. A demo will be used to clarify the concepts.

Eu não sabia que o Elasticsearch fazia isso

Thu, 10 Jun 2021 00:00:00 +0000

O Elasticsearch é muito mais do que um mecanismo de busca full-text. Nesta apresentação exploramos recursos menos conhecidos da plataforma — como agregações, análise de dados, geolocalização e machine learning — mostrando como o Elasticsearch pode resolver problemas que vão muito além da busca tradicional.

OpenTelemetry for Dummies: Instrumenting Go Apps

Thu, 27 May 2021 00:00:00 +0000

OpenTelemetry for Dummies: Instrumenting Go Apps

Wed, 26 May 2021 00:00:00 +0000

Though tracing technologies are not necessarily a new concept only in the recent years it gained enough traction to become one of the key dimensions required to build observability stacks. Undoubtedly the major force behind this traction are the standards created to help developers to collect telemetry data from their applications, such as OpenTelemetry—an observability framework for cloud-native software. OpenTelemetry provides a single set of APIs, libraries, agents, and collectors that ensures technology agnostic collection of traces and metrics—but the implementation for each programming language is different. While Java has an agent capable of automatically instrumenting the JVM with additional bytecode, other programming languages like Go have to handle this instrumentation manually. But Go developers have nothing to fear. This talk will explain in a for-the-rest-of-us style how to instrument applications written in Go and how to send the telemetry data to a backend using a collector.

From Batch to Streams: Building Value from Data In-Motion

Tue, 25 May 2021 00:00:00 +0000

Traditional batch processing forces organizations to wait for data before they can act on it, while modern businesses increasingly need to respond in real time. This talk explains the shift from batch to streaming, showing how treating data as a continuous stream in motion unlocks faster insight, event-driven architectures, and new sources of business value.

Data Streams with Elasticsearch: From Possible to Awesome

Fri, 21 May 2021 00:00:00 +0000

Elasticsearch is often thought of purely as a search engine, but it is also a powerful home for continuously flowing time-series and event data. This session explores data streams in Elasticsearch, showing how to ingest, index, and manage high-volume append-only data efficiently, and how features like index lifecycle management take an ingestion pipeline from merely possible to genuinely awesome.

OpenTelemetry for Dummies: Instrumenting Java Apps

Thu, 20 May 2021 00:00:00 +0000

Tracing has become one of the key dimensions of building observability stacks, and OpenTelemetry has emerged as the standard that lets developers collect telemetry in a technology-agnostic way. This talk explains, in a for-the-rest-of-us style, how to instrument Java applications with OpenTelemetry—including how the Java agent can automatically instrument the JVM with additional bytecode—and how to send the resulting traces and metrics to a backend using a collector.

Hidden Secrets of Instrumenting JVM Apps with OpenTelemetry

Tue, 18 May 2021 00:00:00 +0000

A deep dive into instrumenting JVM-based applications with OpenTelemetry to achieve end-to-end observability. The session reveals the lesser-known details of automatic and manual instrumentation, context propagation, and exporting traces, metrics, and logs, helping developers avoid common pitfalls and gain meaningful visibility into distributed systems.

Metricbeat Modules: Collecting and Visualizing Metrics from Redis

Fri, 07 May 2021 00:00:00 +0000

Using the Metricbeat Redis module to collect and visualize Redis metrics in Kibana.

Metricbeat Modules: How to Speed Things Up with your Deployment

Thu, 06 May 2021 00:00:00 +0000

Tips for speeding up your Metricbeat deployment when working with modules at scale.

Implementing Metric Data Processing: Handling Data with Logstash

Wed, 05 May 2021 00:00:00 +0000

Routing Metricbeat data through Logstash to apply richer transformations before indexing.

Implementing Metric Data Processing: Handling Data with an Ingest Pipeline

Tue, 04 May 2021 00:00:00 +0000

Processing metric data server-side using an Elasticsearch ingest pipeline instead of doing it at the shipper.

Implementing Metric Data Processing: Using Processors with Metricbeat

Mon, 03 May 2021 00:00:00 +0000

How to use Metricbeat processors to enrich, filter, and transform metrics at the edge before shipping them.

Stop being a hero to become a great developer advocate

Mon, 03 May 2021 00:00:00 +0000

If you ask anyone what a developer advocate is, they will probably describe some hero. Someone is often adored by people in conferences with a packed room waiting to hear what they have to say — because they are deemed simply the best in what they do. Almost as if what they do is the result of having superpowers. By all means, I’m afraid I have to disagree with this definition.

Implementing Metric Data Processing: Overview of the Possible Strategies

Sun, 02 May 2021 00:00:00 +0000

An overview of the different strategies available for processing metric data before it reaches its destination.

Getting Started with Metricbeat: Command-Line Options and Outputs

Sat, 01 May 2021 00:00:00 +0000

Exploring Metricbeat’s command-line options and how to configure where your collected metrics are sent.

Getting Started with Metricbeat: Files & Configuration Options

Fri, 30 Apr 2021 00:00:00 +0000

A tour of the Metricbeat configuration files and the options you will most commonly need to tune.

Getting Started with Metricbeat: Downloading, Installing, and Using

Thu, 29 Apr 2021 00:00:00 +0000

A getting-started walkthrough covering how to download, install, and run Metricbeat for the first time.

Metricbeat Introduction: Metricbeat, Modules & Metricsets

Wed, 28 Apr 2021 00:00:00 +0000

An introduction to the core concepts of Metricbeat: how modules and metricsets fit together to gather metrics.

Streaming de dados usando Elasticsearch: do possível ao incrível!

Wed, 28 Apr 2021 00:00:00 +0000

Elasticsearch é conhecido por busca e análise de dados, mas seu potencial vai muito além quando combinado com pipelines de streaming de dados em tempo real. Esta sessão mostra como ingerir e processar fluxos contínuos de dados no Elasticsearch, do cenário mais simples ao mais sofisticado, explorando padrões de arquitetura que permitem indexar, enriquecer e analisar eventos à medida que eles acontecem.

From Zero to Hero with Observability in Microservices

Tue, 27 Apr 2021 00:00:00 +0000

As systems are broken into microservices, understanding what is happening across distributed calls becomes essential and traditional logging alone is no longer enough. This session walks through observability from the ground up, covering metrics, logs, and distributed tracing, and shows how to instrument microservices and correlate telemetry so engineers can quickly detect, diagnose, and resolve issues in production.

Metricbeat Modules: Collecting and Visualizing Metrics from Kafka

Tue, 27 Apr 2021 00:00:00 +0000

How to use the Metricbeat Kafka module to collect broker and topic metrics and visualize them in Kibana.

Everything you Always Wanted to Know about Metricbeat

Mon, 26 Apr 2021 00:00:00 +0000

The kickoff to a full series on Metricbeat, Elastic’s lightweight shipper for collecting metrics from your systems and services.

I Didn't Know that Elasticsearch Could do That

Tue, 23 Mar 2021 00:00:00 +0000

Most people know Elasticsearch as a full-text search engine, but it is capable of far more. This session explores lesser-known features and use cases, from aggregations and analytics to geospatial queries, alerting, and machine learning, revealing capabilities that go well beyond keyword search.

The Best Advice New Developer Advocates Should Follow

Mon, 22 Mar 2021 00:00:00 +0000

There is no denying that the demand for developer advocacy is peaking. Everywhere I go, I can see a huge demand for developer advocates, and this is awesome because this is the career that I chose for myself. But other than this, it is also proof that companies finally realize the value of improving developer experience in the products they offer. As such, this current spike for developer advocates is also a problem. This is not a simple career path to get started with.

Building Data-Driven Applications with Elasticsearch

Sat, 20 Mar 2021 00:00:00 +0000

Elasticsearch is often thought of only as a search engine, but it is equally a foundation for data-driven applications. This talk covers how to model, index, and query data in Elasticsearch, and how to leverage aggregations and its APIs to build responsive applications that put fast search and analytics at their core.

Engenharia de Dados usando Elastic Stack

Wed, 17 Mar 2021 00:00:00 +0000

O Elastic Stack é capaz de trazer grandes benefícios para adeptos e praticantes de engenharia de dados, ainda que seu uso nem sempre seja óbvio, já que muitos acreditam que a plataforma serve apenas para agregar logs. Esta sessão apresenta uma introdução ao Elastic Stack e mostra como ele pode resolver problemas clássicos de engenharia de dados, abordando integração e reconciliação de dados, técnicas de filtragem, agregação e curadoria, além de mecanismos de consolidação de dados para fins analíticos.

How to Help People in Community Forums Like a Boss

Thu, 04 Mar 2021 00:00:00 +0000

Whether it is part of your job description or not, helping users in community forums is always a great idea if you work for a company representing a given technology, product, or service. The key idea is to make sure that users don’t struggle.

But how do you know when and how users are struggling? Well, there are arguably many sources where you can get this information from, such as your support channel, your partners, and your consultancy department, if there is any. Just talk to them periodically and understand where the customers struggle the most. But another one that is becoming fundamental in today’s industry is community forums.

From Zero to Hero with Observability in Microservices

Wed, 17 Feb 2021 00:00:00 +0000

Using Stream Deck with Fedora

Sun, 07 Feb 2021 00:00:00 +0000

I have lost count of how many times I had to apply workarounds in my Fedora laptop to use certain devices. For all of you that had to go down the same path, you know the feeling of hopelessness. The reason why I never get tired of helping people with the challenges I have. This post will describe my journey to get the Elgato Stream Deck device working like a charm on my laptop.

Watch your Streams: Implementing OpenTelemetry with Apache Pulsar

Tue, 26 Jan 2021 00:00:00 +0000

Distributed messaging systems like Apache Pulsar are notoriously hard to observe once messages flow across producers, brokers, and consumers. This session shows how to instrument Pulsar-based applications with OpenTelemetry to capture traces, metrics, and context propagation, giving developers end-to-end visibility into their streaming pipelines.

Elastic Community Conference CfP Clinic

Thu, 07 Jan 2021 00:00:00 +0000

Submitting a talk to a call for papers can be intimidating, especially for first-time speakers. This interactive clinic helps prospective speakers craft strong submissions for the Elastic Community Conference, offering practical guidance on choosing a topic, writing a compelling title and abstract, and framing a proposal that stands out to reviewers.

How to Lose a Microservice in 10 Days

Sat, 05 Dec 2020 00:00:00 +0000

Microservices promise agility and scalability, but the same distribution that makes them powerful can also make them fragile and hard to operate. In this playful yet practical talk, common anti-patterns and mistakes that lead teams to ’lose’ their microservices are explored—from poor service boundaries and chatty communication to missing observability—along with the practices that keep distributed systems healthy and resilient.

Criando Provedores Customizados para Terraform

Thu, 03 Dec 2020 00:00:00 +0000

O Terraform permite descrever e provisionar infraestrutura como código, mas nem toda API ou serviço possui um provider pronto para uso. Esta sessão mostra como escrever um provider customizado para o Terraform em Go, cobrindo a estrutura de um plugin, a definição de recursos e schemas, o ciclo de vida de create, read, update e delete, e como testar e distribuir o provider para que qualquer serviço possa ser gerenciado de forma declarativa.

A Tale about Streaming Analytics using Pulsar and Elastic

Sat, 28 Nov 2020 00:00:00 +0000

A story-driven session that combines Apache Pulsar and the Elastic Stack to build a real-time streaming analytics pipeline. The talk shows how Pulsar ingests and processes high-volume event streams and how that data can be indexed and visualized in Elasticsearch and Kibana to deliver actionable insights with low latency.

Best Practices for Building Streaming Data Architectures

Thu, 26 Nov 2020 00:00:00 +0000

Event streaming has reshaped how organizations move and process data in real time, but designing these architectures well requires more than just adopting a message broker. This talk shares practical best practices for building streaming data architectures with Apache Kafka, covering topic and schema design, data contracts, stream processing patterns, scalability, and the trade-offs involved in evolving from batch to real-time systems.

From Zero to Hero with Observability in Microservices

Tue, 17 Nov 2020 00:00:00 +0000

Getting Started with OpenTelemetry using Elastic APM

Mon, 16 Nov 2020 00:00:00 +0000

OpenTelemetry is an observability framework that provides the libraries, agents, and other components that you need to capture telemetry from your services so that you can better observe, manage, and debug them. It allows you to capture metrics, distributed traces, resource metadata, and logs (logging support is incubating now) from your backend and client applications and then sends this data to backends like Elastic APM.

As you probably have heard of at this point OpenTelemetry is the result of the merge between OpenTracing and OpenCensus, and it aims to provide a more complete, vendor neutral, easy-to-use framework to implement your observability needs.

Observabilidade usando Elastic APM

Tue, 03 Nov 2020 00:00:00 +0000

Observabilidade permite entender o comportamento interno de aplicações distribuídas a partir dos sinais que elas emitem. Esta sessão mostra como usar o Elastic APM para instrumentar serviços, coletar traces, métricas e erros, e correlacionar esses dados no Elastic Stack para diagnosticar problemas de desempenho e latência em produção.

Streaming Data using Apache Pulsar and the Elastic Stack

Fri, 23 Oct 2020 00:00:00 +0000

Apache Pulsar and the Elastic Stack are a powerful combination for building real-time data pipelines. This session shows how to stream events through Pulsar and land them in Elasticsearch, using Kibana to search, analyze, and visualize the data as it arrives, so teams can turn continuous event streams into actionable insights.

Building Data-Driven Apps with Elasticsearch

Thu, 22 Oct 2020 00:00:00 +0000

Elasticsearch is far more than a search engine; it is a foundation for building responsive, data-driven applications. This session introduces the core concepts of indexing, querying, and aggregations, and demonstrates how developers can use Elasticsearch and the Elastic Stack to power features like full-text search, analytics, and real-time dashboards in their own projects.

Building Streaming Data Apps with Pac-Man

Tue, 20 Oct 2020 00:00:00 +0000

In this near-zero-slides talk, the classic Pac-Man game becomes a way to generate events that drive a streaming data application. As the audience plays, a live scoreboard is built from scratch using ksqlDB, the streaming database for Apache Kafka, and the generated data is then sent to Elasticsearch and Kibana where the real action happens.

Criando Provedores Customizados para Terraform

Sat, 17 Oct 2020 00:00:00 +0000

Excluding Files from a List in Terraform

Sat, 10 Oct 2020 00:00:00 +0000

Since the introduction of the for_each feature in Terraform 0.12 it is now possible to code powerful constructs to express the logic of your infrastructure. But sometimes you have to use some creativity to use this feature. I recently had to write a Terraform code that could upload all the JavaScript files from a local folder into a storage blob in Azure. But one of these files would have some template expressions so I wanted to upload all the files from the local folder except that one — because I would upload it individually.

Understanding Streaming Data and Analytics with Apache Kafka®

Fri, 02 Oct 2020 00:00:00 +0000

Slides of the session “Understanding Streaming Data and Analytics with Kafka” that I delivered on ApacheCon 2020. Have fun ✨

https://speakerdeck.com/riferrei/understanding-streaming-data-and-analytics-with-apache-kafka-r

Best Practices for Building Streaming Data Architectures

Wed, 30 Sep 2020 00:00:00 +0000

Building your First Connector for Kafka Connect

Tue, 29 Sep 2020 00:00:00 +0000

A hands-on guide to writing custom connectors for the Kafka Connect framework. The session explains the Connect architecture and the difference between source and sink connectors, then walks through implementing the connector and task APIs, handling configuration and offsets, and packaging and deploying a connector to move data reliably in and out of Apache Kafka.

My Data is Better than Yours! Prioritization in Apache Kafka

Thu, 27 Aug 2020 00:00:00 +0000

Apache Kafka is built around an immutable commit log rather than a traditional message broker, so it has no native concept of message priority the way JMS or AMQP does. This session presents the bucket priority pattern, which groups messages into buckets of partitions at production time and allocates more partitions and consumers to higher-priority buckets, using Kafka’s pluggable custom partitioners and assignors to give important messages a greater chance of being processed first and faster.

Building Event Streaming Applications with Pac-Man

Mon, 24 Aug 2020 00:00:00 +0000

Learning event streaming is a lot easier when it is fun. This session builds a streaming application around the classic Pac-Man game, capturing gameplay events into Apache Kafka topics and using ksqlDB to process them in real time, powering a live scoreboard that aggregates player metrics such as highest score, level reached, and losses as events arrive.

Rediscovering the Value of Apache Kafka® in Modern Data Architectures

Sat, 01 Aug 2020 00:00:00 +0000

Apache Kafka is far more than a message broker; it is the backbone of many modern data architectures. This talk revisits Kafka’s role as an event streaming platform, exploring how it enables real-time data integration, service decoupling, and stream processing across distributed systems.

Troubleshooting 101

Fri, 31 Jul 2020 00:00:00 +0000

Part of my job as Developer Advocate is to reach out to developers in social forums like Stackoverflow, Reddit, and Slack communities to check whether I can provide help. While I feel great pleasure in doing this, it is getting harder and harder to help people these days.

I have the feeling that developers these days don’t know (or perhaps forgot or pretend that they forgot) how to perform basic troubleshooting. Every time I see a request for help in a forum usually people barely explain what the problem is, how it occurs, give evidence of what happened before the incident and, more importantly, to provide a minimal explanation of their environment.

Deep Dive into Kafka's Transactional API

Thu, 30 Jul 2020 00:00:00 +0000

Exactly-once semantics in Apache Kafka rely on the often misunderstood Transactional API. This deep dive explains how transactions, idempotent producers, and the read-committed isolation level work together under the hood, and shows Java developers how to build end-to-end pipelines that neither lose nor duplicate messages, even in the presence of failures and retries.

Apache Kafka with Ricardo Ferreira & Anna McDonald

Fri, 17 Jul 2020 00:00:00 +0000

A live Deploy Friday conversation with Confluent’s Ricardo Ferreira and Anna McDonald exploring Apache Kafka: what event streaming is, why organizations adopt it, and how developers get started building real-time, event-driven applications. The episode covers real-world use cases, common challenges, and practical advice for teams moving toward data in motion.

Data Sharing between Java & Go using Kafka and Protobuf

Thu, 09 Jul 2020 00:00:00 +0000

One of the most incredible things in Apache Kafka is that record values are shapeless, meaning that developers can write any set of bytes they want, and it works just fine. This can be pretty powerful if they share data among different programming languages.

However, this characteristic is sometimes inadequate. It creates coupling problems as producers and consumers need to negotiate a common format that they can rely on to write and read data. Therefore, it is essential to use a neutral format – likely something accepted within the industry, such as Avro, Protobuf, and JSON Schemas. This post will focus on the problems raised by using Protobuf to share data between Java and Go.

Learning Event Streaming Applications with Pac-Man

Wed, 01 Jul 2020 00:00:00 +0000

Pac-Man has been a beacon of fun and joy for people of all ages since its ’80s release, but few realize it can also illustrate event streaming concepts. In this near-zero-slides talk, the game is deployed to a cloud provider directly from a GitHub repository and the audience plays to generate events; during gameplay a scoreboard is built from scratch using ksqlDB, an event streaming technology built out of Kafka Streams, and a serverless API then monitors the aggregated game statistics in near real time to reveal the most skilled Pac-Man player present.

Design Considerations for Kafka Connect Deployments

Fri, 29 May 2020 00:00:00 +0000

Kafka Connect makes it easy to stream data in and out of Apache Kafka, but running it reliably at scale requires deliberate design decisions. This talk walks through the architectural considerations behind Kafka Connect deployments—standalone versus distributed mode, worker sizing and scaling, connector configuration, converters and schemas, error handling, and monitoring—so teams can build robust, maintainable data integration pipelines.

Anti-Patterns for Developing Kafka Streams Apps with Java

Wed, 27 May 2020 00:00:00 +0000

A candid examination of the common mistakes developers make when building Kafka Streams applications in Java. The talk walks through anti-patterns around topology design, state stores, serialization, error handling, and testing, and offers practical guidance and best practices to keep streaming applications correct, performant, and maintainable.

Building Serverless Applications with Confluent Cloud

Tue, 19 May 2020 00:00:00 +0000

Serverless architectures and event streaming are a natural fit: functions react to events while the platform handles scaling and operations. This session shows how to build serverless applications backed by Confluent Cloud, wiring managed Kafka to serverless compute so developers can create responsive, event-driven systems without provisioning or maintaining messaging infrastructure.

Apache Kafka sem Zookeeper: O que muda nas minhas Aplicações?

Mon, 04 May 2020 00:00:00 +0000

O Apache Kafka historicamente dependia do Zookeeper para gerenciar metadados e coordenação do cluster, mas o projeto KRaft passou a incorporar essa responsabilidade ao próprio Kafka. Esta sessão explica o que muda com a remoção do Zookeeper, como o novo modelo de consenso funciona e quais são os impactos práticos na operação e no desenvolvimento das suas aplicações.

Apache Kafka without Zookeeper: What Changes in my Apps?

Mon, 27 Apr 2020 00:00:00 +0000

Apache Kafka has historically relied on ZooKeeper for metadata management, but that dependency is being removed in favor of a self-managed quorum. This session explains what the removal of ZooKeeper means in practice, how the new architecture handles metadata and controller responsibilities, and what developers and operators should expect to change in the way they build and run their Kafka-based applications.

Serializing and Deserializing Kafka Records with Golang

Tue, 07 Apr 2020 00:00:00 +0000

Getting data in and out of Apache Kafka correctly is one of the first challenges Go developers face. This session walks through how records are serialized and deserialized in Golang, covering the trade-offs between formats such as JSON, Avro, and Protobuf, the role of Schema Registry, and how to write producers and consumers that handle evolving schemas safely.

What Apache Kafka has in Common with Rocky Balboa?

Fri, 03 Apr 2020 00:00:00 +0000

If you know me well, you must know that I am a big fan of movies. It is not uncommon to see me in situations where if I need to explain something rather complex, I end up using movies as metaphors to understand what I am trying to explain. In this post, I will explain what Apache Kafka has in common with the character Rocky Balboa.

Rocky Balboa is a fictional character created and played by Sylvester Stallone, who tells the history of a boxer that faces numerous challenges throughout his life and career. But in every single movie of the series (eight produced until this date), it is impossible not to fall in love with the character. This is the first and most important thing that Kafka has in common with this character.

Serializing and Deserializing Kafka Records with Golang

Tue, 31 Mar 2020 00:00:00 +0000

Why the property 'auto.create.topics.enable' is disabled in Confluent Cloud?

Tue, 17 Mar 2020 00:00:00 +0000

If you ever used Apache Kafka you may know that in the broker configuration file there is a property named auto.create.topics.enable that allows topics to be automatically created when producers try to write data into it. That means that if a producer tries to write an record to a topic named customers and that topic doesn’t exist yet – it will be automatically created to allow the writing. Instead of returning an error to the client.

Configuring VPC Endpoints with Confluent Cloud

Thu, 12 Mar 2020 00:00:00 +0000

A hands-on walkthrough of establishing private, secure network connectivity to Confluent Cloud using VPC endpoints and VPC peering. The session explains the networking options available, when to use each, and how to configure endpoints so that Kafka traffic stays off the public internet.

AWS ElastiCache with a Bastion Host using Terraform

Mon, 09 Mar 2020 00:00:00 +0000

AWS ElastiCache is a fully managed service that allows users to easily and quickly use cache technologies like MemCached and Redis without the gory implementation details. But this doesn’t come for free. Developers often complain about the fact that the service is deployed in private subnets and due to that fact – they are not entitled to easily access for troubleshooting purposes.

A pattern to solve this problem is the usage of a bastion host, that sits between your machine and the private subnet to allow troubleshooting with ease. This post will show how to deploy a AWS ElastiCache service for Redis along with a bastion host using Terraform.

Multiple Columns with Redis Sorted Set

Tue, 03 Mar 2020 00:00:00 +0000

Recently I had to build an application that would store and order players in a Redis Sorted Set, but using multiple columns as criteria for the score, instead of just one. After trying to find a proper solution on the web and failing to do so; I decided to put my head to work and figure out an way. This post will show how did I accomplished that.

It turns out that my application has players with the following fields:

Confluent Go Client with Docker

Sat, 29 Feb 2020 00:00:00 +0000

If you are reading this blog post then there is a high chance that you’ve been looking for ways to make Confluent’s Go Client for Apache Kafka work with your Docker images but you’re struggling to. Am I right?

Don’t worry, you’ve came to the right place. After struggling with this myself I decided to share the solution that I’ve found with everybody else so we all can spend more time writing code than just wasting time with dependency plumbing.

Organizations Powered by Apache Kafka and Confluent

Thu, 20 Feb 2020 00:00:00 +0000

Apache Kafka has become the de facto platform for building real-time, event-driven organizations, but running it reliably at scale takes more than the open-source broker alone. This talk explores how companies adopt Kafka to unlock streaming data across their systems, and how Confluent extends it with the tooling, connectors, and managed services needed to move from initial experiments to enterprise-wide event streaming.

Everything you Always Wanted to Know about Kafka * But Were Too Afraid to Ask

Wed, 19 Feb 2020 00:00:00 +0000

Apache Kafka has become the backbone of modern event streaming, yet many developers still have fundamental questions they never get the chance to ask. This session provides a pragmatic, no-judgment tour of Kafka’s core concepts—topics, partitions, producers, consumers, offsets, and delivery guarantees—demystifying how the platform works and how to reason about it when building real applications.

Using Logitech Products with Fedora Linux

Thu, 06 Feb 2020 00:00:00 +0000

I am a huge fan of what Logitech creates for the computer industry and I have been using its products for quite some time now. However, for the last 5 years I took the [smart] decision of using Linux for desktop and by doing so – I realized that certain products seems only to work well with Mac/Windows, leaving no space for Linux.

In this post I will provide some tips about open-source projects that helps you to get your favorite Logitech products working smoothly with Linux, Fedora in particular.

Everything you Wanted to Know about Apache Kafka but you Were too Afraid to Ask!

Thu, 28 Nov 2019 00:00:00 +0000

An honest, myth-busting tour through Apache Kafka that answers the questions developers often hesitate to raise. The talk clarifies how Kafka really works under the hood, from the commit log and partitioning to delivery guarantees and consumer offsets, so attendees leave with a solid mental model and the confidence to build event streaming applications correctly.

Rediscovering the Value of Apache Kafka in Modern Data Architectures

Tue, 26 Nov 2019 00:00:00 +0000

A look at how Apache Kafka has evolved from a simple messaging system into the backbone of modern data architectures. The session explores Kafka’s role as a central nervous system for streaming data, showing how event-driven patterns, stream processing, and connectors enable organizations to unify operational and analytical workloads in real time.

Building Event Streaming Applications with Serverless Kafka

Mon, 18 Nov 2019 00:00:00 +0000

Event streaming is at the heart of modern cloud-native systems, but operating Kafka clusters can distract teams from building features. This session explores how a serverless approach to Kafka lets developers build event streaming applications without managing brokers, using fully managed infrastructure that scales on demand, and how this model fits naturally into Kubernetes and cloud-native architectures.

You Too Can Build Event-Driven Apps with Apache Kafka

Wed, 06 Nov 2019 00:00:00 +0000

Event-driven architectures decouple services and let systems scale and evolve independently, and Apache Kafka has become the de facto backbone for building them. This session introduces the core building blocks of Kafka and walks through how to design and implement event-driven applications that produce, consume, and process streams of events.

Redescobrindo o Papel do Apache Kafka® em Arquiteturas de Dados Modernas

Sat, 02 Nov 2019 00:00:00 +0000

Muito além de uma simples fila de mensagens, o Apache Kafka se tornou a espinha dorsal de arquiteturas de dados modernas. Esta apresentação revisita o papel do Kafka como plataforma de streaming de eventos, mostrando como ele viabiliza integração de dados, processamento em tempo real e desacoplamento entre serviços em sistemas distribuídos.

Building Event Streaming Apps using Pac-Man

Wed, 30 Oct 2019 00:00:00 +0000

Learning event streaming is far more fun when the demo is a game everyone recognizes. Using a playable Pac-Man built on Apache Kafka, this session shows how player events flow through topics, get processed with stream processing, and feed a real-time scoreboard, turning abstract streaming concepts into something you can see and interact with.

Being an Apache Kafka Developer Hero in the World of Cloud

Tue, 29 Oct 2019 00:00:00 +0000

Running Apache Kafka in the cloud changes how developers design, deploy, and operate event streaming applications. This talk covers the tips, tricks, and architectural practices that let Java and other developers be productive with Kafka as a managed cloud service, avoiding common pitfalls around connectivity, scaling, and cost.

Kafka Summit 2019 (San Francisco)

Fri, 04 Oct 2019 00:00:00 +0000

This week I had the honor of being one more time in the most exciting conference for Apache Kafka in the world, known as Kafka Summit. Only this time was different because I could contribute a lot for this conference.

My first contribution was building and recording a demo that was shown in the second day of the conference when Priya Shivakumar was talking about Confluent Cloud. I have received some good feedback about that demo, which of course made me feel really proud.

Kafka Summit San Francisco 2019

Tue, 01 Oct 2019 00:00:00 +0000

Session delivered at Kafka Summit San Francisco 2019.

Oracle Code One 2019

Mon, 23 Sep 2019 00:00:00 +0000

Last week I had the chance to attend one more time Oracle’s new developer’s conference know as Oracle Code One. It was so nice to catch up with great friends, plus being able to party with people from the Java community. They certainly know how to make things funny and entertaining.

Furthermore, I also had the chance to share a little about Apache Kafka in a talk that I delivered for a room full of developers interested in this technology.

Keeping your Data Close and your Caches Hotter

Wed, 18 Sep 2019 00:00:00 +0000

Distributed caches bring data closer to the CPU to speed up execution and enable horizontal scaling, but keeping cached data consistent becomes hard once multiple applications write directly to the underlying database, leaving caches stale and forcing teams into extra code that hurts agility. This session presents cache-based architecture patterns that keep caches always hot using Apache Kafka and its connectors, demonstrating them across in-memory data grids such as Hazelcast, Apache Ignite, and Coherence and across topologies including cache-aside, read-through, write-behind, and refresh-ahead.

Palestra "Abraçando a Anarquia: Design Patterns de Apache Kafka em Arquiteturas de Dados Modernas e Big Data" no QCon São Paulo

Sun, 15 Sep 2019 00:00:00 +0000

Esse ano eu tive a honra de poder palestrar pela primeira vez no QCon São Paulo , edição do evento que ocorreu em Maio de 2019.

Na ocasião, eu fiz parte da trilha de Big Data coordenada pela fera Fabiane Nardon, onde pude compartilhar um pouco da história do Apache Kafka , qual seu estado atual como tecnologia, e, quais casos de uso ele ajuda a resolver.

Vídeo :

https://www.youtube.com/watch?v=4ONf63koLtU

Slides :

https://speakerdeck.com/riferrei/abracando-a-anarquia-design-patterns-de-apache-kafka-em-arquiteturas-de-dados-modernas

My Talk @ Kafka Summit SFO 2019

Mon, 09 Sep 2019 00:00:00 +0000

We are less than one month away from the next edition of the world’s best conference about Apache Kafka known as Kafka Summit, and I would like to detail a bit what is going to be the focus of my talk Being an Apache Kafka Developer Hero in the World of Cloud.

https://kafka-summit.org/sessions/developer-hero-cloud/

As you can read in the description, this talk is all about Confluent Cloud, where I will be showing the first steps to start working with event streaming applications using this awesome Apache Kafka as-a-service product from Confluent. However, the interesting part here is how I am going to do that.

Building Event Streaming Apps with Pac-Man

Sat, 24 Aug 2019 00:00:00 +0000

Pac-Man has been a beacon of fun for people of all ages since the ’80s, but few realize it can also teach event streaming. In this near-zero-slides talk, the game is deployed to a cloud provider straight from a GitHub repository, the audience plays to generate a stream of events, and a live scoreboard is built from scratch with ksqlDB on top of Kafka Streams, with a serverless API monitoring the aggregated stats in near real time to crown the most skilled player in the room.

Mastering Apache Kafka on Kubernetes using Confluent Operator

Thu, 22 Aug 2019 00:00:00 +0000

Running Apache Kafka in production on Kubernetes introduces operational challenges around stateful workloads, storage, scaling, and lifecycle management. This talk explores how the Confluent Operator automates the deployment and day-two operations of Kafka and the broader Confluent Platform on Kubernetes, covering configuration, scaling, and best practices for running a resilient cluster.

Mastering Apache Kafka on Kubernetes using Confluent Operator

Fri, 16 Aug 2019 00:00:00 +0000

Everything you Wanted to Know about Apache Kafka but you Were too Afraid to Ask!

Thu, 25 Jul 2019 00:00:00 +0000

Tips and Tricks about Apache Kafka in the Cloud for Java Developers

Mon, 22 Jul 2019 00:00:00 +0000

Running Apache Kafka in the cloud changes how Java developers design, deploy, and tune their event streaming applications. This session shares practical tips and tricks for building efficient Kafka producers and consumers, configuring clients for managed cloud environments, and avoiding the common pitfalls that trip up teams moving their workloads off self-managed clusters.

Tips and Tricks about Apache Kafka in the Cloud for Java Developers

Thu, 27 Jun 2019 00:00:00 +0000

Implementing Distributed Tracing ‘Like a Boss’ in your Kafka Deployments

Tue, 18 Jun 2019 00:00:00 +0000

Because Kafka producers and consumers are decoupled and unaware of each other, understanding how a single request flows across an event-driven system can be surprisingly hard. This talk shows how to instrument Kafka-based applications with distributed tracing, propagating context through producers, brokers, and consumers so you can visualize dataflows end to end, pinpoint latency, and debug production issues with confidence.

Everything you Wanted to Know about Apache Kafka but you Were too Afraid to Ask!

Tue, 04 Jun 2019 00:00:00 +0000

Keeping your Data Close and your Caches Hotter

Mon, 03 Jun 2019 00:00:00 +0000

Tips and Tricks about Apache Kafka in the Cloud for Java Developers

Wed, 29 May 2019 00:00:00 +0000

Tips and Tricks about Apache Kafka in the Cloud for Java Developers

Tue, 28 May 2019 00:00:00 +0000

JEEConf, Kiev, Ukraine

Wed, 15 May 2019 00:00:00 +0000

This year I had the honor of speaking at JEEConf; one of the most important Java conferences in Europe hosted in the heart of Ukraine. The conference held two days full of content related to Java and related technologies.

I was really impressed with the number of smart people that I could shake hands and meet at the conference. One of the sessions that I presented there was Tips & Tricks about Apache Kafka in the Cloud for Java Developers, that aimed to show some of the details that developers building Java applications for Apache Kafka should know before venturing themselves in moving their workloads to Cloud.

Abraçando a Anarquia: Design Patterns de Apache Kafka em Arquiteturas de Dados Modernas e Big Data

Mon, 06 May 2019 00:00:00 +0000

Arquiteturas de dados modernas trocam o controle centralizado por sistemas distribuídos e orientados a eventos, o que exige novos padrões de projeto. Esta sessão apresenta design patterns de Apache Kafka para desacoplar serviços, processar fluxos de dados em tempo real e integrar plataformas de Big Data, mostrando como abraçar essa ‘anarquia’ de forma organizada e escalável.

Implementing Distributed Tracing ‘Like a Boss’ in your Kafka Deployments

Sat, 27 Apr 2019 00:00:00 +0000

Tips and Tricks about Apache Kafka in the Cloud for Java Developers

Fri, 26 Apr 2019 00:00:00 +0000

GCP for Apache Kafka Users: Stream Ingestion and Processing

Tue, 09 Apr 2019 00:00:00 +0000

A practical look at how Apache Kafka users can build streaming ingestion and processing pipelines on Google Cloud Platform. The session maps familiar Kafka concepts to GCP services such as Pub/Sub, Dataflow, and BigQuery, showing patterns for moving and transforming data in motion at scale.

Keeping Your Data Close and Your Caches Hotter

Tue, 02 Apr 2019 00:00:00 +0000

Modern event-driven applications increasingly need low-latency access to the data flowing through Apache Kafka, without hammering an external database on every request. This session shows how to keep your data close to your services by using Kafka Streams and its state stores to build local, queryable caches that stay continuously updated from your topics, turning Kafka into the backbone of fast, resilient, materialized views.

Getting Superman X-Ray Vision: Bringing Observability to your Stream Processing

Wed, 27 Mar 2019 00:00:00 +0000

Microservices have become the de-facto way to build complex information systems by breaking the domain problem into multiple reusable services, but this distribution makes it far harder to understand what is actually happening across a stream processing pipeline. This talk shows how to gain X-ray vision into event streaming applications by applying observability techniques such as distributed tracing, metrics, and logging so you can see how data flows through your services and pinpoint where problems occur.

Fundamentals of Apache Kafka for Beginners

Wed, 20 Mar 2019 00:00:00 +0000

A practical introduction to Apache Kafka for developers new to event streaming. This session covers the core concepts behind Kafka’s distributed log architecture, including topics, partitions, producers, consumers, and consumer groups, and explains how these building blocks come together to power scalable, fault-tolerant data pipelines.

Fundamentals of Apache Kafka for Beginners

Tue, 05 Mar 2019 00:00:00 +0000

Building Cloud-Native Apps with Confluent Cloud

Tue, 05 Feb 2019 00:00:00 +0000

Building applications on top of Apache Kafka no longer requires standing up and operating your own clusters. This session shows how to design cloud-native, event-driven applications backed by Confluent Cloud, a fully managed Kafka service, covering how to connect producers and consumers, offload operational burden to the platform, and scale elastically so developers can focus on business logic rather than infrastructure.

Building a Streaming Data Pipeline with Kafka Streams and KSQL

Mon, 04 Feb 2019 00:00:00 +0000

Streaming data pipelines let applications react to events as they happen instead of waiting for batch jobs to run. This session shows how to build an end-to-end pipeline on Apache Kafka, using Kafka Streams to transform and enrich events in code and KSQL to express the same stream processing declaratively with SQL-like queries.

LinkedIn and the Nightmare of Connection Requests

Mon, 07 Jan 2019 00:00:00 +0000

Is this just me or LinkedIn became this annoying social network where recruiters and people you never worked with keep poking you for new connections?

Like other professional social networks, LinkedIn was designed to connect people around the globe and empower them with the ability to get jobs, hire talents and be the backbone for professional communication. However, that concept has been exploited because there are people that leverage this concept for their own benefits.

#1 Apache Kafka Meetup in São Paulo/Brazil

Tue, 11 Dec 2018 00:00:00 +0000

During the week of Dec 2th I had the pleasure of visiting São Paulo/Brazil, a place that I once called home. It has been almost 5 years that I left that city to live abroad – and every time I go there its inevitable not to miss certain things.

I went down there to participate of the Google Cloud Summit event, and make sure that developers using GCP would be aware of Confluent’s Apache Kafka as a service know as Confluent Cloud. It was amazing to share this information with them, since most of them was working (or planning to work) with huge amounts of data in the Cloud and Kafka is certainly the right way to do it.

Confluent Cloud Tools is Alive

Fri, 09 Nov 2018 00:00:00 +0000

It gives me great pleasure to announce that Confluent Cloud Tools is officially released as a Confluent’s repository. The link to this repository is:

https://github.com/confluentinc/ccloud-tools

In a nutshell, the aim of this project is to allow developers to unfold the true agility that Confluent Cloud can provide –allowing them to focus on writing code instead of wasting time with manual and tedious configuration steps.

Very often developers use tools from Confluent such as Schema Registry, REST Proxy, Connect, KSQL and Control Center – along with their Kafka clusters. This may work fine when they are working On-Premise and making use of the integrated experience that Confluent Platform provides – whether if it is running Confluent Platform locally and managing things via the CLI or; if it is by using the built-in Docker containers.

ThinkPad Thunderbolt 3 Dock | Suspend Problem

Thu, 11 Oct 2018 00:00:00 +0000

As mentioned in this post; I have acquired a new laptop from Lenovo, the ThinkPad X1 Extreme. Since I will be working from home sometimes, I thought it would be a good idea to buy an dock station for it.

So I decided to buy the following dock station:

ThinkPad Thunderbolt 3 Dock

It is a nice dock station solution, especially because it connects everything to the laptop via Thunderbolt 3. However; since I am running Fedora in my laptop, things couldn’t be that easy right?

New Lenovo ThinkPad X1 Extreme

Tue, 09 Oct 2018 00:00:00 +0000

“It’s not a laptop. It’s a ThinkPad.” - I really like this slogan because it helps to explain why I took the decision to buy the new ThinkPad X1 Extreme from Lenovo, while there are some good options out there - presumably better ones. But let’s start from the beginning.

I have been working with IT for quite some time now, and for some reason I have never had a laptop of my own. Instead, I always used the laptops given by the companies I worked for. This is not a bad decision per-se, but in the long run you may find yourself disappointed with the whole keep-moving-things-around every time you change jobs.

Getting Myself Certified in AWS

Wed, 29 Aug 2018 00:00:00 +0000

August 2018 was a very productive month to me, where after some intense dog hours studying like crazy and practicing a lot - I was able to get certified in two of the most interesting AWS exams: Solutions Architect and Developer Associate.

That was quite of a challenge to me, because my Cloud background had been up to that point mainly focused on Oracle Cloud. It was interesting to see the differences between the two Cloud vendors, as well as seeing how AWS structures its services. Which BTW is amazing.

About

Mon, 01 Jan 0001 00:00:00 +0000

To Vector, or not to Vector, that is the Question

Mon, 01 Jan 0001 00:00:00 +0000