• Test Data Management

    Get more information from your test data!

PEAK ODS ADAPTER FOR APACHE SPARK

Your engineering data. Ready for scalable analytics.

Unlock the value of measurement data already managed in ASAM ODS. Connect it to Apache Spark, notebooks, enterprise analytics platforms and Big Data Lakes — without rebuilding your engineering data foundation.

Open standards   ·   Spark SQL & DataFrames  ·  Parquet / Avro / JSON  · Scale as needed


FROM ENGINEERING DATA TO ANALYTICS VALUE

Stop rebuilding data integration for every new analysis.

Test data is often rich in technical meaning but difficult to use outside its original engineering environment. The Peak ODS Adapter for Apache Spark connects domain-specific ASAM ODS data management with familiar scalable data technologies.

↔

Reuse your data foundation

Access measurements and metadata from existing ODS environments rather than creating another parallel engineering repository.

⌕

Get directly to relevant data

Use Spark SQL and DataFrames to select and process the engineering information needed for each analysis.

▥

Scale with demand

Grow from local analytics to distributed processing with appropriately configured Spark resources and licenses.

◇

Connect big-data ecosystems

Export defined data selections in standards-oriented formats that fit existing lake and analytics workflows.

ONE ENGINEERING DATA FOUNDATION · TWO INTERGRATION PATHS

Access data directly — or prepare it for a Data Lake.

Choose the integration pattern that fits your workload. Both paths start with the same well-described ASAM ODS measurement data.

Path 01 · Live integration

Direct access for Spark analytics

Query ODS instance data and measurement values with Spark SQL and DataFrames. Work with selected signals in Python, Jupyter or other Spark-based applications, using scalable resources when needed.

  • Interactive and repeatable explorative analysis
  • Selective access to measurement datasets
  • Notebook-based collaboration and downstream processing
Path 02 · Standardized export

BigODS Exporter for Data Lakes

Prepare selected engineering data as files optimized for the target big-data environment. The BigODS Exporter is part of the Peak ODS Adapter for Apache Spark — not a separate standalone platform.

  • Configurable export definitions
  • Metadata and instance data in JSON or Avro
  • Measurement mass data in Parquet; optional associated content

INCLUDED COMPONENT  · PEAK BIS ODS EXPORTER

Move the right data to the lake. Not every file by default.

Enterprise analytics does not always require copying complete test campaigns. The integrated BigODS Exporter lets teams define what to export, preserve technical context, and supply analytics environments with structured datasets.

01

Select relevant data

Use XML-based Export Definition Files to describe the ODS data subset required for a specific workload.

02

Preserve its meaning

Include engineering metadata and relationships alongside measurement values, with optional additional files such as media or documents.

03

Deliver analytics-ready formats

Export metadata and instance data as JSON or Avro, and measurement mass data as Parquet for downstream processing.

Apache ParquetApache AvroJSONASAM ODS Big Data Connector

  

WHERE TEAMS PUT IT TO WORK

From domain-specific testing to enterprise analytics.

∿

Cross-test data exploration

Analyze selected time-series channels and engineering metadata from existing ASAM ODS repositories using SQL, Python and notebooks.

▦

Enterprise data platforms

Connect a trusted engineering data foundation to Spark-based enterprise analytics, without duplicating core TDM functionality.

⇢

Data Lake integration

Export purpose-defined ODS datasets to open big-data formats for broader data engineering and analytical use.

APPLICATION EXAMPLE  · GLOBAL AUTOMOTIVE MANUFACTURER

PeakTDM meets Databricks: An engineering data platform built for reuse.

In a global automotive implementation, Peak Test Data Manager provides an ASAM ODS-based, multi-domain engineering data foundation. The Peak ODS Adapter for Apache Spark connects this foundation to Databricks for scalable enterprise processing and analytics.

LAYER 3 · ANALYTICS & AIDatabricks

Scalable enterprise analytics, Spark-based processing and AI development capabilities.

↕
LAYER 2 · DATA INTEGRATIONPeak ODS Adapter for Apache Spark

Reusable access to ASAM ODS metadata and measurement data through Spark SQL and DataFrames.

↕
LAYER 1 · ENGINEERING DATA FOUNDATIONPeak Test Data Manager · ASAM ODS

Domain-specific metadata models, test data ingestion, validation and controlled access across engineering disciplines.

Instead of reconstructing ingestion, metadata management and engineering-tool integration for every new analytics initiative, the customer relies on a reusable data foundation with a dedicated integration layer.

The integration is an implemented customer architecture. Examples such as anomaly detection, predictive engineering and AI assistants are possible future applications, not claimed delivered project results.

ANSWERS BEFORE YOU GET STARTED

Common questions

Do we need to move all measurement data into a new platform?

No. Direct Spark access can work with the existing ODS data infrastructure. For Data Lake use cases, the BigODS Exporter can export selected datasets, avoiding indiscriminate duplication. The exact deployment depends on storage access and the customer architecture.

Is Peak BigODS Exporter included in the Spark Adapter?

Yes. Peak BigODS Exporter is a functional part of the Peak ODS Adapter for Apache Spark and implements the ASAM ODS Big Data Connector export approach.

Can we connect to Databricks?

Yes. PeakTDM has been integrated with Databricks in a global automotive customer architecture via the Peak ODS Adapter for Apache Spark. The specific Spark, network, authentication and deployment setup is project dependent.

Does this only work for automotive test data?

No. The adapter operates on ASAM ODS data. Its applicability depends on the available ODS data model and the requirements of your testing or simulation domain.

Can we start without a large Spark cluster?

Yes. The solution can be used with local Spark resources and scaled to more powerful clusters as needs grow, subject to licensing and infrastructure requirements.

Ready to unlock more value from your ODS data?

Tell us about your current data infrastructure and analytics goals. We will explore the most suitable integration route — direct Spark access, configurable data-lake exports, or a combination of both.

Discuss your requirements ↗

Connected solutions

One data foundation. Connected engineering workflows.

Discover the components and technologies that help you build a complete, standards-based test data environment.

Insights & Practical Knowledge

Explore More on This Topic

Discover related articles, practical insights, and additional solutions to explore this topic in greater depth. Learn more about relevant technologies, real-world applications, and how they can benefit your business.