Martini Flat File Functions

Overview

Martini Flat File Functions let you read data from and write data to flat files — including delimited, fixed-width, and Excel formats — by returning memory-efficient cursors. These functions integrate directly into your services and workflows, enabling row-by-row file processing without loading entire files into memory.

What You Will Learn

• How to open an input cursor to read records from a flat file
• How to open an output cursor to write data to a flat file

When To Use This

Use this in advanced scenarios instead of a predefined Flat File Service when you need runtime-configurable processing of large flat files (delimited, fixed-width, or Excel).

Prerequisites

• Martini Designer installed and running
• A Martini package in the current workspace
• A service or workflow to invoke the functions from
• Basic understanding of the following:
  • Invoke Function Node for workflows
  • Function Step for services

Reading Flat Files with Flat File Functions

Martini provides open*FileInputCursor functions that open a read-only cursor backed by an InputStream, allowing your service or workflow to process flat file data one row at a time.

The available input reading functions are:

• openDelimitedFileInputCursor(...) — Reads delimited flat files such as CSV and TSV
• openFixedWidthFileInputCursor(...) — Reads fixed-width flat files
• openExcelFileInputCursor(...) — Reads Excel spreadsheets (.xls, .xlsx)

The following sections use openDelimitedFileInputCursor as the primary example. The cursor concepts described here apply equally to all input functions.

Getting Started with Reading a Delimited Flat File

This example shows how to read customer records from a CSV file using openDelimitedFileInputCursor in a workflow.

Sample delimited flat file (customers.csv)

id,name,email,city,country
1,Alice Johnson,alice.johnson@example.com,New York,USA
2,Bob Smith,bob.smith@example.com,London,UK
3,Carol White,carol.white@example.com,Sydney,Australia

Build the reading workflow

flowchart LR
    s[...]
    a["`**Function Node:** Open delimited flat file input cursor and map to _myCursor_`"]
    b["`**Repeat Node:** Iterate over _myCursor_`"]
    c["`Process the current _myCursor_ entry`"]
    d[...]
    s --> a --> b -- each --> c
    b -- then --> d

Step 1: Configure the Function Node

In the Function Node, invoke openDelimitedFileInputCursor and provide at a minimum:

• flatFileData — the InputStream of the flat file to read
• fieldNames — an ordered list of property names to assign to each cursor entry

openDelimitedFileInputCursor(
    flatFileData: myInputStream,
    fieldNames: ['id', 'name', 'email', 'city', 'country']
)

Map the returned cursor to a data model array named myCursor.

Step 2: Configure the Repeat Node

In the Repeat Node, set myCursor as the input array. For each iteration, myCursor holds one row from the file with the properties you defined in fieldNames.

Expected result: Each row of customers.csv is available as a named cursor entry without loading the entire file into memory:

myCursor {
    id = 1
    name = Alice Johnson
    email = alice.johnson@example.com
    city = New York
    country = USA
}

Shaping the Flat File Input Cursor

The shape of each entry in the input cursor — the property names available on each row — is determined by what you pass to fieldNames, modelReferenceName, or modelTemplate. If none of these are provided, Martini generates property names automatically.

Property order matters

Cursor entries are populated positionally: the first column maps to the first property, the second column to the second property, and so on. When using fieldNames, modelReferenceName, or modelTemplate, ensure that the property order matches the column order in the flat file; otherwise, values may be assigned to the wrong properties without generating an error.

Default behavior (no shaping configured)

Without fieldNames, modelReferenceName, or modelTemplate, Martini assigns sequential names starting from field0:

myCursor {
    field0 = 1
    field1 = Alice Johnson
    field2 = alice.johnson@example.com
    field3 = New York
    field4 = USA
}

This is useful for quick inspection but impractical for production pipelines where meaningful property names are required.

Using fieldNames

Pass an ordered list of names to fieldNames.

openDelimitedFileInputCursor(
    flatFileData: myInputStream,
    fieldNames: ['id', 'name', 'email', 'city', 'country']
)

Result:

myCursor {
    id = 1
    name = Alice Johnson
    email = alice.johnson@example.com
    city = New York
    country = USA
}

Using modelReferenceName

To reuse an existing data model, pass its fully qualified namespace to modelReferenceName. Martini uses the model's properties as the cursor entry structure.

openDelimitedFileInputCursor(
    flatFileData: myInputStream,
    modelReferenceName: 'my_package.CustomerModel'
)

Using modelTemplate

Pass a populated data model instance to modelTemplate. Martini uses the non-null properties of the instance to define the cursor entry structure. This is helpful when you have a pre-built data model at runtime and want the cursor to reflect it without specifying a namespace manually.

How Flat File Reading Works

Martini's Flat File reading functions open a cursor backed by the incoming InputStream. Rather than buffering the full file in memory, the cursor reads and exposes one row at a time as your workflow or service iterates.

Reading flow:

1. Call openDelimitedFileInputCursor (or another open*FileInputCursor function) with an InputStream.
2. Martini configures the cursor using the input properties you provided — delimiter, encoding, field names, and so on.
3. The function returns a read-only cursor mapped to a data model array.
4. Iterate over the cursor using a Repeat Node or Iterate Step. Martini loads each row into the cursor's current entry, then discards it and loads the next after processing.
5. After all rows are processed, the cursor closes automatically.

Flat File Reading Benefits and Use Cases

Martini Flat File reading functions solve the challenge of processing large files without exhausting system memory. Common use cases include:

• Bulk data import: Read thousands of customer, product, or transaction records from a CSV file and insert them into a database.
• ETL pipelines: Extract data from a flat file source, transform each row, and load it into another system.
• Report processing: Parse fixed-width report exports from legacy systems and map them to Martini data models.
• Excel ingestion: Read structured data from .xls or .xlsx spreadsheets delivered by external partners.

Troubleshooting Flat File Reading Issues

Problem	Detection	Cause	Fix
Cursor returns no entries	Repeat Node completes without processing any rows	InputStream is empty or already consumed before being passed to the function	Open a fresh InputStream positioned at the start before calling the input cursor function
Cursor entry properties are null or misnamed	Data model fields are null after mapping	fieldNames list does not match the actual column order in the file	Verify that each name in fieldNames corresponds to the correct column index in the flat file
Column values appear under the wrong property names	Cursor entries contain swapped or unexpected values	fieldNames order does not align with the column sequence in the file	Reorder fieldNames to match the exact column sequence in the flat file
Garbled or unreadable character values	Text values contain unexpected symbols or boxes	File encoding does not match the encoding property passed to the function	Set the correct character encoding in the encoding input property — for example, UTF-8
First data row is missing after iteration	Record count is one less than expected	The firstRow property is set to treat the first row as a header when it contains data	Adjust the firstRow input property to match how your file's first row is actually structured

Writing Flat Files with Flat File Functions

Martini provides open*FileOutputCursor functions that open a write-only cursor backed by an OutputStream, allowing your service or workflow to write records to a flat file incrementally.

The available output writing functions are:

• openDelimitedFileOutputCursor(...) — Writes delimited flat files such as CSV and TSV
• openFixedWidthFileOutputCursor(...) — Writes fixed-width flat files
• openExcelFileOutputCursor(...) — Writes Excel spreadsheets (.xls, .xlsx)

The following sections use openDelimitedFileOutputCursor as the primary example. The cursor concepts described here apply equally to all output functions.

Getting Started with Writing to a Delimited Flat File

This example shows how to write customer records to a CSV file using openDelimitedFileOutputCursor in a workflow.

Build the writing workflow

flowchart LR
    s[...]
    a["`Prepare _customers_ data model array`"]
    b["`**Function Node:** Open delimited flat file output cursor and map to _myCursor_`"]
    c["`**Repeat Node:** Iterate over _customers_ with _myCursor_ as the output array`"]
    d["`**Map Step**: Map the current _customers_ entry to the current _myCursor_ entry`"]
    e[...]
    s --> a --> b --> c -- each --> d
    c -- then --> e

Step 1: Configure the Function Node

In the Function Node, invoke openDelimitedFileOutputCursor and provide at a minimum:

• flatFileData — the OutputStream to write the flat file to
• fieldNames — an ordered list of property names that define the output row structure

openDelimitedFileOutputCursor(
    flatFileData: myOutputStream,
    fieldNames: ['id', 'name', 'email', 'city', 'country']
)

Map the returned output cursor to a data model array named myCursor.

Step 2: Configure the Repeat Node

In the Repeat Node, set customers as the input array and myCursor as the output array. For each iteration, map the current customers entry to the current myCursor entry. Martini appends each mapped entry to the output cursor, serializing it to the OutputStream.

Expected result: The output CSV file contains one row for each entry in customers:

1,Alice Johnson,alice.johnson@example.com,New York,USA
2,Bob Smith,bob.smith@example.com,London,UK
3,Carol White,carol.white@example.com,Sydney,Australia

Shaping the Flat File Output Cursor

Like input cursors, the structure of each entry in the output cursor is shaped by fieldNames, modelReferenceName, or modelTemplate. This determines which properties each output row entry exposes.

Property order matters

Output rows are written positionally: the first property maps to the first column, the second property to the second column, and so on. When using fieldNames, modelReferenceName, or modelTemplate, ensure that the property order matches your intended output column sequence; otherwise, values may be written to the wrong columns.

Default behavior (no shaping configured)

Without fieldNames, modelReferenceName, or modelTemplate, the output cursor accepts any properties you append to it.

Using fieldNames

Pass an ordered list of names to fieldNames. The output cursor entry exposes exactly those properties, and each appended entry is serialized as a row in the same sequence.

openDelimitedFileOutputCursor(
    flatFileData: myOutputStream,
    fieldNames: ['id', 'name', 'email', 'city', 'country']
)

If you append:

myCursor {
    id = 1
    name = Alice Johnson
    email = alice.johnson@example.com
    city = New York
    country = USA
}

The row written to the file is: 1,Alice Johnson,alice.johnson@example.com,New York,USA

Using modelReferenceName or modelTemplate

The same concepts from Shaping the Flat File Input Cursor apply here. Provide modelReferenceName with the fully qualified namespace of a data model, or provide modelTemplate with a data model instance that has at least one non-null property.

How Flat File Writing Works

Martini's Flat File writing functions open a cursor backed by the outgoing OutputStream. As your workflow appends entries to the output cursor, Martini serializes each row and writes it to the stream incrementally.

Writing flow:

1. Call openDelimitedFileOutputCursor (or another open*FileOutputCursor function) with an OutputStream.
2. Martini configures the cursor using the input properties you provided — delimiter, encoding, field names, and so on.
3. The function returns a write-only cursor mapped to a data model array.
4. In the Repeat Node or Iterate Step, set the source data array as the input and the output cursor as the output array.
5. For each entry in the source data, map the values to the current cursor entry. Martini serializes the entry and writes it to the OutputStream, then advances to the next entry.
6. After all entries are written, the cursor closes automatically.

Flat File Writing Benefits and Use Cases

Martini Flat File writing functions enable you to generate flat files incrementally, without buffering the entire output in memory. Common use cases include:

• Data export: Export records from a database query to a CSV or TSV file for consumption by external systems.
• Report generation: Produce fixed-width report files expected by legacy downstream systems.
• Excel output: Generate .xlsx spreadsheets from application data for end-user consumption.
• Batch file delivery: Write large datasets to flat files as part of scheduled batch jobs.

Troubleshooting Flat File Writing Issues

Problem	Detection	Cause	Fix
Output file is empty	File exists but contains no rows	The source data array was empty when the Repeat Node ran	Verify the source data array (for example, customers) is populated before the output cursor function is called
Output file contains misaligned or shifted columns	Consuming system cannot parse the file	Properties are appended to the cursor entry in a different order than fieldNames expects	Map source data to cursor entry properties in the exact order defined by fieldNames
Wrong values appear in output columns	Column values do not match the expected data	fieldNames order does not match the intended output column sequence	Reorder fieldNames to match the intended column sequence in the output file
Output delimiter does not match the target format	Consuming system cannot split rows correctly	The delimiter input property was not set or was set to the wrong character	Set the correct delimiter in the delimiter input property — for example, , for CSV or \t for TSV

Helpful Resources

• Flat File Services — Configure reusable flat file structure definitions for static, repeatable file processing
• Flat File Types — Understand the differences between delimited, fixed-width, and Excel file formats
• Martini Cursors — Deep dive into how cursors work and how to manage them efficiently
• Invoke Function Node — Call Flat File Functions from within a workflow
• Function Step — Call Flat File Functions from within a service
• Community Q&A: Martini Community
  Have a question? Post or search it here.