Integrate filesystem

HULFT Integrate provides its own implementation of a filesystem abstraction layer. The purpose of this abstraction layer is to allow for Integrate Services to access different types of concrete filesystems in a uniform and platform independent way without having to know what type of filesystem they're accessing.

Concept of Integrate filesystem

Platform independent
Regardless of which platform a script is made, whether it is UNIX/Linux or Windows, it can be executed on whichever platform the script is loaded,
because the filesystem abstraction layer offers the flexibility to change the platform a script is executed, without any modification to the implementation of file handling operations in a script.

ACL
Integrate filesystem provides access control list for file and directory access privileges, thereby it is possible to protect against tamper or deletion if multiple users work simultaneously.

Data consistency
Compared to ordinary files, the risk of inconsistency occurring or data loss is reduced by the use of database whose integrity provides the validity and consistency of stored data.

Integrate filesystem architecture

Integrate filesystem architecture is represented in the diagram below.

Component Layer

Component layer consists of components which interact with Integrate filesystem's abstraction layer.
These components include Integrate Studio, CLI Console, and adapter components which fall into File adapter category.
All of these components perform indirect access to the filesystem through Integrate filesystem's abstraction layer via its specified interface between the component layer and the platform filesystem or a database.

Integrate filesystem Layer

Integrate filesystem Layer provides a common file model that is capable of representing conceivable filesystem's general features and behavior.
There are three types of Integrate filesystem, each controls data flow between the component layer and the physical layer by its own specific way.

Physical Layer

This is the layer that contains the actual data.
It could be the platform dependent filesystem or a database, or both, depending on how Integrate filesystem is configured.

Types of Integrate filesystem

There are 3 different types of Integrate filesystem.

filesystem Types	Description	Notation used in [Type] attributes in Explorer
Local filesystem	Data is stored in the OS's filesystem.It performs the fastest, but it cannot save the file meta-information such as owner and permissions of a file, or whether a file is write-protected by the file lock feature or not. If there is no repository database used, Local filesystem it the only option available for the filesystem type.	LOCAL
Extended local filesystem	The file meta-information is stored in the repository database and data itself is stored in the OS's filesystem.It does not perform as fast as the Local filesystem. Can be used for the system with a repository database.	LOCAL\|EXTENDED
Database filesystem	Both the file meta-information and the file data are stored in the repository database table. the maximum file size supported varies depending on the DBMS used. Can be used for the system with a repository database.	none

Integrate type of filesystem is synonymous with the type of mount.

OS local filesystem

OS local filesystem is a term we use to distinguish the actual filesystem of the OS from the aforementioned abstraction filesystem layer provided by Integrate filesystem.

There are two terms we use for this OS local filesystem, accordingly to which part of the HULFT Integrate is running, whether it is IntegrateServer or Integrate Studio.

Server OS local filesystem
- Refers to the filesystem of the OS in which IntegrateServer is running
Client OS local filesystem
- Refers to the filesystem of the OS in which Integrate Studio is running
- Refers to the filesystem of the OS in which Integrate Studio for Web is running

Mount

Mount is a mechanism used to associate the Integrate filesystem with a particular point in the OS local filesystem's hierarchy or a database table. The associated filesystem directory in the OS local filesystem's hierarchy is called mount point

The association of a mount point and Integrate filesystem can be done in Mount Settings in the Control Panel.

System Mount

There are number of directories in the OS's filesystem's hierarchy associated with the Integrate filesystem by default when the system starts up:
System mount point is as follows.

When there is use of a repository database

Database filesystem
- /etc
- /home
- /share
Extended local filesystem
- /data
- /ftp
- /testreport
Local filesystem
- /samples
- /schema
- /tmp

When no repository database is used

Local filesystem
- /data
- /etc
- /ftp
- /home
- /samples
- /schema
- /share
- /testreport
- /tmp

The directories for system mount can be changed in system.properties.

They cannot be changed if database filesystem is used.

Restrictions on characters used in Integrate filesystem

Use of the following characters and strings are disallowed in Integrate filesystem.

\ / : ; * ? ' " < > | ~

Platform dependent characters and external characters

Use of characters and symbols that are not common in both Windows and UNIX/Linux platform are restricted.
Additionally, the tilde sign, ~ is disallowed as it represents a shot cut that points to a user's home directory in some platform.

Case sensitivity

The following table presents how in Integrate filesystem case sensitivity plays a part.

Item	Filesystem Type	case sensitive or case insensitive	applicable fields
Files and directories found under /etc, /home, /share (when a repository database is used)	Database filesystem	Complies with the database you use and settings of that	Project name Service name Global Resource name Global Schema name Trigger name Holiday settings calendar name In the file path string to the log output destination. In the name of a user or a group(when a repository database is used)
Files and directories found under /etc, /home, /share (no repository database is used)	Local filesystem	Depends on OS platform in which IntegrateServer is running Windows version: Case insensitive UNIX/Linux version: Case sensitive
Name of files and directories to be stored, except for /etc,/home,/share	Local filesystem and Extended local filesystem	Depends on OS platform in which IntegrateServer is running Windows version: Case insensitive UNIX/Linux version: Case sensitive	-
Names stored in configuration files.	-	Case sensitive	Script names Folder names Script variable names Environment variable names Names of icons Names for log search criteria
Names of files and directories stored under associated mount points	Database filesystem	Case Sensitive	-
	Local filesystem and Extended local filesystem	Depends on OS platform in which IntegrateServer is running Windows version: Case insensitive UNIX/Linux version: Case sensitive	-

It is encouraged to be mindful of case sensitivity issues when developing application in order to avoid case-related problems when performing server migrations to different OS platforms in the future.

Specifications may change depending on the version.

Specification Limitations

Limitations in mount operations
- Directories that are immediate children of the root "/" directory can only become mount points.
  Directories that are not associated with a mount point will be deleted at restart.
- It is discouraged to use the following directories as mount points.
  - /home,/home/<username>
  - Directories found under /home/<username>/projects and /home/<username>/projects
  - /etc and directories found under /etc
  - /share and directories found under /share
- A directory which is specified as a mount destination needs to have its parent directory.
- If you have booted IntegrateServer using the Windows service and would like to mount a directory in the network drive, specify the user which has access permission to the network directory as an account name when you log on to the Windows service. After rebooted IntegrateServer, you need to specify the directory path in the network drive in a UNC name format to the mount source.
No file can be created immediately under the '/' root directory of the Integrate filesystem.

Any relative file path set in a settings of a component is relative to the directory that is selected in the file and directory Selectr or in Explorer.
It only affects the execution of the client tools such as designer .

The consistency of the data written to the same file by multi-threaded operation is not guaranteed.

Depending on the environment, when OS local filesystem is referenced by the directory Selectr from Studio, a dialog box warns about 'missing disk' may be prompted.
This can be ignored and the process is continued by pressing [Continue] button.

This issue has been confirmed, when Integrate is hosted by Windows running on VMware(not always)