× Install ThecoreGrid App
Tap below and select "Add to Home Screen" for full-screen experience.
B2B Engineering Insights & Architectural Teardowns

FHIR and Kafka for Wearable Analytics

Cloud-native architecture for wearables addresses not only the ingestion challenge. It must accommodate heterogeneous devices, low latency, FHIR requirements, and clinical access control.

The main engineering challenge here is not data collection, but the fact that the wearable stream immediately encounters three incompatible requirements. On one hand, it is necessary to accept high-frequency vital signs in real time. On the other hand, they must be converted to a common medical format. And all this must be done without compromising interoperability, security, and clinical workflow requirements.

The authors propose a pragmatic solution: instead of integrating directly with each device, data is collected through health frameworks of mobile platforms. This reduces the maintenance burden and eliminates dependence on vendor-specific APIs. The stream is then divided into microservices, Kafka, and stream processing, with FHIR becoming the canonical exchange format. The compromise is clear: the architecture becomes more complex, but in return gains scalability, layer isolation, and independent development of ingestion, analytics, and storage.

A key part of the scheme is the transformation pipeline. Raw measurements first pass through a mobile application built on Flutter, which utilizes Apple HealthKit and Google Health Connect. Then, the Import Service on FastAPI validates the JWT, checks the JSON schema, and sends messages to Kafka. On the processing side, Kafka Streams Mapper breaks down batches into individual measurements, normalizes them into FHIR, and performs local validation. Here, there is an important engineering fork: full FHIR validation is CPU-intensive, so the system allows for sampling validation through an environment variable. This is not an ideal option, but it allows for managing the cost-quality trade-off while maintaining throughput under load.

The storage strategy deserves special attention. Storing full FHIR in JSON is costly, especially when data arrives as high-frequency time series. Therefore, the authors introduce a dependency-aware minimization scheme: the system stores only the minimal set of values from which the full FHIR resource can be reconstructed without data loss. To achieve this, a dependency tree is built based on declarative YAML mapping, and then the data is converted to InfluxDB Line Protocol and stored in InfluxDB. This is a compromise between medical semantics and storage costs. The complete FHIR structure is not lost, but a more compact representation is maintained in the working contour.

The architecture is not limited to the hot path. For analytics and ML, the medallion lakehouse pattern is implemented through Spark Structured Streaming. The Bronze layer stores raw messages, the Silver layer normalizes them into Delta Lake, and the Gold layer provides features to Feast and supports model serving through MLflow. This important separation is because clinical decision support and retrospective research have different requirements for latency, durability, and reproducibility. In one contour, the system serves real-time inference, while in another, it handles research workloads. Thus, the architecture avoids mixing latency-critical operations with heavy batch tasks.

At the presentation layer, the platform delivers data through REST, GraphQL, and Grafana. This is not just a set of interfaces, but a way to differentiate various access models. Clinicians need a visual overview and case context. Researchers require exports and queries by time, patient, device, and measurement type. Grafana is used as the primary layer for visualizing time-series data, while the data management service serves as the boundary between the frontend and storage. This approach reduces coupling and simplifies the development of interfaces without rewriting lower layers.

From a compliance perspective, the system relies on RBAC, secure service-to-service communication, and the ability to deploy in a private cloud. This is important because in healthcare, the architecture must be not only functional but also manageable in terms of access and data sovereignty. The authors specifically note that the stack is built on open-source software, making it transparent for institutional deployment.

The results show where the system has real capacity and where bottlenecks arise. The ingestion pipeline handles 50 full ingestion requests per second with median response times below 8 ms. This is a good signal for real-time patient monitoring. For ML inference, throughput increases linearly to about 980 line-protocol messages per second, while processing a single inference task takes about 10–15 seconds due to the micro-batch nature of Spark Structured Streaming. In the hot path, higher loads reveal bottlenecks in the Kafka Streams Mapper and Telegraf. As the load increases to 30 messages per second, there is no noticeable degradation, but beyond that, latency rises sharply. This is not a failure of the architecture, but an honest boundary of the current configuration, indicating where further optimization will be needed.

If we view the system as an engineering artifact, its strength lies elsewhere. It does not attempt to make wearables “simple.” Instead, it accepts their real complexity and breaks it down into layers: acquisition, transformation, storage, analytics, presentation. This approach may not be flashy, but it is the one that typically works in a clinical environment.


Information source

arXiv is the largest open preprint repository (since 1991, under the auspices of Cornell), where researchers quickly post working versions of papers; the materials are publicly accessible but do not undergo full peer review, so results should be considered preliminary and, where possible, checked against updated versions or peer‑reviewed journals. arxiv.org

View the original research PDF

×

🚀 Deploy the Blocks

Controls: ← → to move, ↑ to rotate, ↓ to drop.
Mobile: use buttons below.