Starting with Database Version 11g, Oracle offers with Direct NFS an integrated NFS-Client, which is independent from the underlying OS-based NFS-Client. This allows direct access to the NFS storage for the Oracle Instance and realizes Oracle’s own optimizations. In addition, one can implement multipathing over four parallel ports, independently from any switches, NIC bonding or teaming. The Oracle Instance configures the communication with the NFS automatically and thus facilitates the configuration under RAC. Using Oracle Direct NFS, shortly dNFS, also provides using NFS Storage with Oracle Instances on Windows systems.
The Direct NFS Client supports NFS Server using NFSv3 with database version 11g and from 12c on NFSv3 to NFSv4.1. Before Direct NFS can be used, it must be enabled and set up at least for a Single Instance. For that two requirements need to be fullfilled. First the Direct NFS ODM (Oracle Disk Manager) Library must be loaded at the strat of the Instance and second the mount point and NFS storage needed to be configured. Following the configuration tasks with Linux and Windows:
1. Loading Direct NFS ODM Library
Using Linux the Direct NFS Client had to be activated.
Linux: Login as Oracle software installation owner
> cd $ORACLE_HOME/rdbms/lib > make -f ins_rdbms.mk dnfs_on
This option is only supported since 11.2.
Manually, you can do also the following:
> cd $ORACLE_HOME/lib > mv libodm11.so libodm11.so.stub > ln -s libnfsodm11.so libodm11.so
Using Windows the standard ODM Library, oraodm11.dll had to be replaced by ODM NFS Library, oranfsodm11.dll in the bin Directory.
Windows: Shutdown the database and change into ORACLE_HOME\bin directory
> copy oraodm11.dll oraodm11.dll.stub > copy /Y oranfsodm11.dll oraodm11.dll
At the start of the database following lines should be found in the Alert Log for both Linux and Windows:
Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 3.0
2. Configurating mount point and NFS storage
a. Mount point configuration
To create the mount points using Linux in general it is sufficient to configure the /etc/fstab or /etc/mtab. In addition the oranfstab can be created to use Oracle specific options. The Direct NFS Client determines the settings of the mount point(s) in succession and if available
Linux: fstab mount options – You should give “deafult” a chance. Are there any problems the key entry is the NFS buffer size parameters rsize and wsize. Oracle recommends a value of 32768. Using RAC the entry in the fstab should be the same on every node.
nas1:/volume1/oranfs /mnt/oranfs nfs rw,bg,hard,rsize=32768,wsize=32768,nfsvers=3,nointr,timeo=600,actimeo=0,tcp,addr=172.16.0.2 0 0
Linux: Optional oranfstab (minimum information bold)
server: nas1 # alias freely selectable path: 172.16.0.2# IP NFS server local: 172.16.30.40 # optional IP database server path: # optional up to 4 path/local pairs for multipathing local: # optional multipathing for load balancing and failover export/volume1/oranfs/oradata mount: /mnt/oranfs/oradata # export: NFS server path, mount: local path export: mount: # optional additional export/mount paths - unlimited mnt_timeout: 30 # optional timeout when mounting fails dontroute # optional messages should not be routed by OS nfs_version: nfsv4.1 # optional from 12c + NFSv4 (default: NFSv3) management # optional from 12c use management interface for SNMP community # optional from 12c community string for SNMP server: nas2 # additional server seperated by blank line path: 172.16.0.3 export: /volume/path/nfs mount: /local/path
In opposite to Linux the oranfstab has to be used with Windows. The Direct NFS Client determines the mount options only using ORACLE_HOME\dbs\oranfstab. Accordingly the oranfstab file has to be created manually (as textfile without file extension). The specified mount path (here: D:\oranfs\oradata) hasn’t to be exisit on the Oracle-Server. If you specify the path the Oracle Instance automatically detects that it is a NFS path and puts the file on the NFS server. Using RAC the oranfstab has to be created identically on all nodes.
Windows: oranfstab (minimum information bold)
server: nas1 # alias freely selectable >path: 172.16.0.2 # IP NFS server local: 172.16.30.40 # optional IP database server path: # optional up to 4 path/local pairs for multipathing local: # optional multipathing for load balancing and failover export: /volume1/oranfs/oradata mount: D:\oranfs\oradata # export: NFS server path, mount: local path export: mount: # optional additional export/mount paths - unlimited mnt_timeout: 30 # optional timeout when mounting fails uid: 65534 # optional User ID for NFS server access gid: 65534 # optional Group ID for NFS server access nfs_version: nfsv4.1 # optional from 12c + NFSv4 (default: NFSv3) server: nas2 # additional server seperated by blank line path: 172.16.0.3 export: /volume/path/nfs mount: D:\local\path
b. NFS storage configuration
The NFS server must at least support NFS version 3.
Espacially with Linux-based servers, there is a problem with port reservation, because the Oracle Instance in not running as root user. It is therefor necessary to prevent rejection by the NFS server on this by setting the insecure option in the /etc/exports. This allows connections to non-priviliged ports above 1024.
nas1> cat /etc/exports /volume1/oranfs *.carajandb.intra(rw,async,no_wdelay,insecure,root_squash,insecure_locks,sec=sys,anonuid=1024,anongid=100)
To ensure, creating a tablespace on the NFS server works correctly, the permission should be set correctly.
nas1> cd /volume1/oranfs > chown oracle:dba oradata > chmod 755 oradata
Windows: The Direct NFS Client uses by default ui:65534 and gid:65534, which should correspond with nobody and nogroup on the NFS server. The access should be ensured but can in doubt be set in the oranfstab. For the right permissions you can create an user oracle and a group ORA_DBA on the NFS server and set the permissions accordingly:
nas1> cd /volume1/oranfs > chown oracle:ORA_DBA oradata > chmod 755 oradata
First a query to check if the NFS server is connected:
select * from v$dnfs_servers; D SVRNAME DIRNAME MNTPORT NFSPORT WTMAX RTMAX -- ------------ ------------------------- ------- ------- ------ ------ 1 nas1 /volume1/oranfs/oradata 892 2049 131072 131072
If the configuration is not correct, the query simply returns no rows. If the instance already was restarted it is also possible that the path must addressed first before the query returns a result. Create a file on the mount for testing and repeat the query:
SQL> create tablespace test_tbs datafile 'D:\oranfs\oradata\test_tbs.dbf' size 10M;
SQL> create tablespace test_tbs datafile '/mnt/oranfs/oradata/test_tbs.dbf' size 10M;
If the configuration runs, you can determine if the access to the NFS really runs over Direct NFS by using the following query. Returns the query no rows the NFS-Client of the operating system is used at Linux. Help is only given by the Alert Log und Trace files.
select * from v$dnfs_files; FILENAME FILESIZE PNUM SVR_ID -------- -------- ---- ------ d:/oranfs/oradata/test_tbs.dbf 10493952 10 1
More information can be retrieved with following views:
select * from v$dnfs_channels; select * from v$dnfs_stats;
Linux: A possible source for problems could be the wrong permission on oradism. The Direct NFS Client needs root privileges for initializing the communication with the NFS storage. In this case the owner is not root but oracle. The Alert Log shows up with the following error message:
Direct NFS: please check that oradism is setuid Direct NFS: warn could not mount /volume1/oranfs/oradata on svr 172.16.0.2 via dNFS
> ls -l $ORACLE_HOME/bin/oradism -rwxr-x--- 1 oracle oinstall > chown root:root oradism > chmod 4755 oradism
Generally the permissions look like the following:
> ls -l $ORACLE_HOME/bin/oradism -rwsr-x--- 1 root oinstall