47. Overview UPDF Files

updf (Uni-Plot-Data-Format) will be the new file format for UniPlot 6.0. It will be used for UniPlot document files (ipw files) and netCDF files (nc files).

In the current UniPlot version only netCDF-up files are supported. To enable the new data format choose toolsmore-optiones and select netCDF-up (compressed) for nc-File Type.

UniPlot 6.0 will still be able to create and read the original netCDF format. Both formats are using the same netCDF interface. A flag will indicate the new format during file creation.

47.1. Format Description

updf files are set up like file systems. An updf file contains directories. The directories can contain files and directories. As in a file system, files can be added, removed, renamed, extended or moved.

Directories and files can contain attributes, so called key-value pairs. Different data types, e.g. Unicode strings or 64-bit integers, are supported. Scalar values, vectors and matrices of values can be saved.

To avoid confusion directories are named storages, files are named streams and attributes are named properties. Properties have a property name and a property value.

47.2. Advantages of the new Format

• Writing UniPlot 5.x files can be very slow for big files with multiple pages. Writing of updf files is faster: The time to write a file increases linearlly, i.e. to write 200 pages takes only twice as long as 100 pages.
• The current ipw files and nc files do not use compression. Updf uses compression and thus data is saved compactly.
• Current nc files are limited to 2 Giga-Bytes in size. Updf files are limited to 8 Tera-Bytes.
• updf-nc files can be embedded in updf-ipw-files.
• nc-files encounter performance problems if data is modified, e.g. if a global attribute has been added to a file, the complete nc file must be written to disk. This is not necessary for updf-nc-files.
• Transactions are possible for updf-nc-files.

47.3. Format Properties

• Max. size of updf files: approx. 8 Terabytes (8000 GB).
• Number of characters for storage-, stream- and property names: 255.
• Max. size of streams: 8 TB.
• Max. number of streams and storages per storage: 2^31. In real life some 10,000.
• Max. size of a property value: Some MB’s (must fit in RAM memory).
• Number of properties per stream or storage: 2^31. In real life some 100,000.

47.4. Valid Names in Storages, Streams and Properties

A valid names contains 1 to 255 Unicode characters.

The following characters are invalid: Characters in in range 0 to 31 and the following 9 characters ‘/’, ‘’, ‘*’, ‘?’, ‘”’, ‘<’, ‘>’, ‘:’, ‘|’.

47.5. Logging

Normally updf function errors are written in UniPlot’s Event Window. With the command

log_set_active("updf", 0)

logging can be turned off.

47.6. Garbage collection

Garbage collection is used for all pointers. Unclosed elements will be closed in the correct order by the garbage-collector (gc).

47.7. UniScript Interface

	Tools
updf_dump	updf_dump.
updf_fileversion	updf_fileversion returns the file version.
updf_gen	updf_gen copies the all files of directory an its subdirectories into an updf file.
updf_get_error_string	updf_get_error_string returns the error message for the error code returned by updf_is_error.
updf_is_error	updf_is_error checks if a fatal error occurred.
updf_libversion	updf_libversion.

	Storages and Streams
updf_close_storage	updf_close_storage closes a storage of sub storage.
updf_commit	If a root storage is given the changes are written to disk.
updf_copy_to	updf_copy_to copies a storages and all containing streams, storages and properties recursivly into another storage.
updf_create_file	updf_create_file creates a new updf file.
updf_create_storage	updf_create_storage creates a new storage within a storage.
updf_create_stream	updf_create_stream creates a new stream in a storage.
updf_destroy_element	updf_destroy_element removes (destroys) a storage or stream in a storage.
updf_open_file	updf_open_file opens an existing updf file.
updf_open_storage	updf_open_storage opens a storage in a updf file.
updf_open_stream	updf_open_stream opens an existing stream in a updf file.
updf_rename_element	updf_rename_element changes the name of a stream or storage.
updf_revert	updf_revert reverts all modifications since the last open or commit command (updf_commit).

	Streams
updf_create_stream	updf_create_stream creates a new stream in a storage.
updf_file_close	updf_file_close closes a updf stream.
updf_file_copy_to	updf_file_copy_to.
updf_open_stream	updf_open_stream opens an existing stream in a updf file.

	Iteration (Storages and Streams)
updf_enum_begin	updf_enum_begin can be used with updf_enum_next to iterate over all streams and storages of a storage object.
updf_enum_close	updf_enum_close
updf_enum_next	updf_enum_next can be used with updf_enum_begin to iterage over all streams and storages of a storage object.
updf_stat_get_len	updf_stat_get_len is the size of a stream (number of bytes). If stat contains a storage object the function returns 0.
updf_stat_get_name	updf_stat_get_name returns the storage of stream name.
updf_stat_get_type	updf_stat_get_type returns the type of an element.

	Properties
updf_prop_close	updf_prop_close closes the properties in a storage or stream.
updf_prop_copy_to	updf_prop_copy_to.
updf_prop_count	updf_prop_count returns the number of properties.
updf_prop_delete	updf_prop_delete removes a property-object (key/value pair).
updf_prop_enum	updf_prop_enum iterates over all properties of a stream or storage.
updf_prop_enum_close	updf_prop_enum_close closes a propEnum object.
updf_prop_enum_next	updf_prop_enum_next can be used with updf_prop_enum to iterate over all properties of a stream and storage.
updf_prop_get_at	updf_prop_get_at returns a property-object.
updf_prop_has_key	updf_prop_has_key checks if the given property-name exists.
updf_prop_open	updf_prop_open opens the properties of a given storage or stream.
updf_prop_set_at	updf_prop_set_at writes a property.

	Property-Values
updf_propval_get	updf_propval_get returns a property value.
updf_propval_get_info	updf_propval_get_info returns information about a property.
updf_propval_new	updf_propval_new creates a new property value object.

See also

Functions by Categories

_id-824069