Using DBFS – Oracle Database File System

DBFS Architecture
DBFS Architecture

The Oracle Database File System (DBFS) is a feature that creates a standard file system interface on top of files and directories that are stored in database tables. DBFS is similar to NFS in that it provides a shared network file system that looks like a local file system. Like NFS, there is a server component and a client component.

The database provides much better security, availability, robustness, transactions, and scalability than traditional file systems. When files are stored in the database, they are backed up, synchronized to the disaster recovery site using Data Guard, and recovered along with the relational data in the database. This has made storing files in the database an appealing option for such applications.

In DBFS, the server is the Oracle Database. Files are stored as SecureFile LOBs in a database table. A set of PL/SQL procedures implement the file system access primitives such as create, open, read, write, and list directory. The implementation of the file system in the database is called the DBFS Content Store. The DBFS Content Store allows each database user to create one or more file systems that can be mounted by clients. Each file system has its own dedicated tables that hold the file system content.

Pre-requisites

Before you begin, you need to check that FUSE has been installed on the server. For details on how to check this and  manually install FUSE see my previous article Installing FUSE on linux (for DBFS)

Creating a DBFS filesystem (in the database)

First, create a tablespace to hold the filesystem:

CREATE TABLESPACE dbfs_goldengate datafile size 1G autoextend on next 1G;

Create a user to own the filesystem:

CREATE USER dbfs_admin IDENTIFIED BY dbfs_jnsdhiehosnjdh
TEMPORARY TABLESPACE temp
DEFAULT TABLESPACE dbfs_goldengate
QUOTA UNLIMITED ON dbfs_goldengate;
GRANT create session, resource, create view, dbfs_role TO dbfs_admin;

Create the filesystems:

connect dbfs_admin
@?/rdbms/admin/dbfs_create_filesystem.sql DBFS_GOLDENGATE TESTDB1

You can check that the filesystems were created successfully with the following SQL:

col created for a25
col store_mount for a30
SELECT store_mount, store_name, created
FROM table(dbms_dbfs_content.listmounts);

​dbfs_create_filesystem.sql creates a partitioned file system.

Partitioning is the best performing and scalable ​​way to create a file system in DBFS. Multiple physical segments are created in the database and files are distributed randomly in these partitions. However, space cannot be shared between partitions. So it is possible for one partition to run out of space even when other partitions have space. This is usually not an issue if file system size is big compared to the size of the individual files. However if file sizes are a big percentage of the file system size, it can cause ENOSPC error to happen even if the file system is not full. Another implication of partitioning is that a "rename" operation can require the file to be rewritten. This can be expensive if file is a big file.

dbfs_create_filesystem_advanced.sql can be used to create a non-partitioned file system.​​​​

Mounting the filesystem at the OS level

On each server where the filesystem may be mounted, create an empty directory (mount point) to attach the DBFS filesystem to:

mkdir -p /oracle/goldengate
chown oracle:oinstall /oracle/goldengate

Testing that the filesystem can be mounted

The following command will prompt for the dbfs_admin password and then mount the filesystem, but note that the session will "hang" until the filesystem is un-mounted by another terminal session.

dbfs_client dbfs_admin@dbfs /oracle/goldengate

So, in another session check that the TESTDB1 filesystem was successfully mounted and  that you can write to it.

mount | grep dbfs

ls -l /oracle/goldengate
touch /oracle/goldengate/TESTDB1/test_file
ls -l /oracle/goldengate/TESTDB1
rm /oracle/goldengate/TESTDB1/test_file

Finally, to un-mount the filesystem you must use the fusermount command as the umount command will report the error "umount: /oracle/goldengate mount disagrees with the fstab")

fusermount -u /oracle/goldengate

Configure the filesystem to automount via fstab and Oracle Wallet

Before continuing, ensure that you have created and configured an Oracle Wallet for accessing the dbfs_admin database user. See the previous article "A guide to Oracle Wallet" for details on how to do this.

For the examples below, we have created a wallet and configured a TNS entry of  "DBFS" to access the dbfs_user via the wallet.

cat >>/etc/fstab <<EOF
#DBFS Goldengate Filesystem
/sbin/mount.dbfs#/@dbfs /oracle/goldengate fuse wallet,rw,user,noauto,allow_other 0 0
EOF

Because we are using the allow_other mount option we also need to add the "user_allow_other" option to the fuse configuration file /etc/fuse.conf. This will allow users other than the owner to be able to see the filesystem.

cat >>/etc/fuse.conf <<EOF
user_allow_other
EOF

Testing that the filesystem can be mounted via fstab

The following command should be run as oracle as this is the user that will be writing the goldengate files to the DBFS filesystem.

mount /oracle/goldengate

Provided no errors are reported, you should confirm that the TESTDB1 filesystem was mounted and that file operations can be performed as follows:

mount | grep dbfs

ls -l /oracle/goldengate
touch /oracle/goldengate/TESTDB1/test_file
ls -l /oracle/goldengate/TESTDB1
rm /oracle/goldengate/TESTDB1/test_file

Finally, to un-mount the filesystem you must use the fusermount command as the umount command will report the error "umount: /oracle/goldengate mount disagrees with the fstab")

fusermount -u /oracle/goldengate