Skip to the yard
Waxsealai

Flint-interactive Gives AI Agents a Higher-Level Language for Interactive Charts

Shipped
Shipper
Chenglong Wang, Yuwei Xiao, Alper Sarikaya, Michel Galley, Chris Quirk, Jianfeng Gao
Readers
696

Flint-interactive maps user selections to chart updates, helping agents build linked visualizations and dashboards with less low-level rendering work.

Flint-interactive Gives AI Agents a Higher-Level Language for Interactive Charts
Flint Interactive flowchart | purple gradient background

Interactive charts can turn an AI-generated report into a space for exploration: readers can select data, compare views, or use a chart selection to focus related text. Building those behaviors reliably is still costly. Flint-interactive addresses that gap with a higher-level language for describing what a user does and how a visualization should respond.

Illustration of Doug Burger

Why interaction is difficult to generate

AI agents can analyze data and produce visualizations, but a static chart limits readers to the view chosen by its creator. Adding custom interactions often means repeated rounds of code generation, reasoning, and debugging. That takes time and tokens, and changes meant to fix one issue can introduce another, such as a broken interaction or poor layout.

Built-in interactions in visualization libraries cover common needs, but may not express the bespoke behavior a chart requires. Flint’s premise is that agents need a language that describes interactions at a useful level of abstraction, while leaving the low-level work of connecting events to rendered changes to a compiler.

From selections to linked updates

Flint-interactive builds on Flint to support interactive charts, dashboards, data stories, and other connected artifacts. A user or agent specifies interaction logic through semantic events and update operators. The compiler interprets those specifications and manages the visual dependencies involved in applying an update.

For example, selecting a group of countries in one Gapminder scatterplot can highlight the same countries in a second view showing a different year. Flint offers a preset called linked-brush for this pattern. The selection can help a reader compare how the chosen group relates across views without requiring a developer to hand-code each chart’s rendering details.

Figure: Flint interactive allows the agent to describe interaction logic by mapping “what semantic event is the user interacting with” to “what updates they want for the canvas surface”. The compiler takes the heavy lifting of resolving visual updates and manages the runtime, which makes interaction authoring more reliable and cost effective.
Interactive Gapminder visualization showing linked brushing across two scatterplots of countries in 1952 and 2007. Each plot charts GDP per capita on the horizontal axis and life expectancy on the vertical axis, with countries colored by continent. A blue rectangular selection highlights a group of countries in the 1952 chart, and the same countries are simultaneously highlighted in the 2007 chart, demonstrating linked selection across years. A control panel on the right provides interaction options including rectangular selection, lasso selection, and x- or y-axis brushing. The figure illustrates how selecting data points in one view highlights corresponding points in another view for comparative analysis over time.
A linked-brush interaction carries a selection from one chart into another, supporting comparison across years.

Three ways to use Flint-interactive

For chat or canvas workflows, Flint-interactive can be used through Flint-MCP and MCP Apps, following the setup guide. A user can ask an agent for a chart and specify desired interactions; the agent can produce an interaction_spec alongside a Flint chart specification and render the result in the conversation. Users can then select part of the chart and carry that context into a follow-up turn.

graphical user interface, chart
In a chat environment, an agent can generate an interactive chart from Flint specifications and pass a user’s selection into a follow-up turn.

For application builders, Flint Interactive Canvas can power charts inside a dashboard, data story, or other app. Event handlers receive semantic events from the canvas and can use them to filter a table, update another chart, or change application state. An app can also send update operators to the canvas so its own widgets control chart behavior.

Interactive Gapminder dashboard for 2007 showing a table of countries alongside a bubble chart comparing GDP per capita and life expectancy. In the table, the row for the United States is selected and highlighted, displaying GDP per person of $42,952 and life expectancy of 78.2 years. The corresponding bubble on the scatterplot is emphasized and labeled with a callout showing the same values. Countries are colored by continent, bubble size represents population, GDP per capita is shown on a logarithmic horizontal axis, and life expectancy is plotted on the vertical axis. The figure demonstrates external selection, where choosing a country in a table highlights and annotates the matching data point in the visualization.
Selecting a country in a table can highlight and annotate its corresponding point in a chart, connecting the table and visualization.

For interactions beyond the presets, developers can define a trigger and custom handler through Flint’s programming API. A handler can respond to a semantic event by combining operators such as set-style to highlight a mark and set-annotation to label it. That keeps the interaction focused on the intended result instead of requiring direct manipulation of renderer-specific marks.

Interactive climate visualization showing a seasonal phase portrait for five U.S. cities, with temperature (°C) on the horizontal axis and precipitation (mm/day) on the vertical axis. Seattle is selected, and its annual weather cycle is displayed as a connected path labeled by month from January through December. The path shows cool, wet conditions in winter, peaking near 6 mm/day of precipitation, and warmer, drier conditions in summer, with precipitation dropping below 1 mm/day. Colored points represent Chicago, Houston, Los Angeles, New York, and Seattle. A details panel on the right displays the selected month, January, with a temperature of 4.2°C and precipitation of 5.7 mm/day. The figure demonstrates temporal dragging and animation, where moving a city’s position along its seasonal path updates all cities to the corresponding time of year.
A bespoke interaction lets a user drag a city along its seasonal path and see temperature and precipitation change over time.

How the interaction model works

Flint Canvas acts as a state machine, tracking user input and the canvas’s visual state. Its compiler translates raw actions, such as clicks, drags, and key presses, into semantic events that identify the selected chart elements and data. It also translates update operators into canvas changes, handling hit testing and other lower-level rendering work.

The user defines the interaction logic by mapping those semantic events to updates. Operators can change a mark’s style, add an annotation, or adjust the viewport for panning and zooming. The same events can drive updates outside the chart, including changes to tables, text, widgets, or application state. This separation makes interaction specifications easier to maintain because authors work with the meaning of a selection rather than the rendering machinery behind it.

Flint’s compositional event and update model also provides a way to connect different parts of an application. A chart can respond to a table selection, while a chart selection can update a paragraph or another view. For teams building data stories and dashboards, that offers a shared interaction model across the visualization and its surrounding interface.

Where developers can start

Chart users can follow the Flint chat setup guide and try examples from the interaction gallery. App builders can consult the interaction API to add connected behavior to dashboards and data stories. Designers can explore bespoke interaction examples and develop reusable presets.

The practical test for Flint-interactive will be whether its higher-level specifications reduce iteration while still giving teams enough control over the behavior they need. Its design targets a clear source of friction: translating a user’s action into consistent updates across charts and application elements. That makes it a useful approach to evaluate when an agent-generated visualization needs to support exploration beyond a single static view.

Source: Chenglong Wang, Yuwei Xiao, Alper Sarikaya, Michel Galley, Chris Quirk, Jianfeng Gao · www.microsoft.com

Same vesselRelated cargo