Domino 12 Backup & Restore

Daniel Nashed
HCL Lifetime Ambassador
 Agenda
• Why Domino 12 native Backup

• Main Features and basic setup

• Linux, S3 and other options

• Snapshot Backup in Domino 12.0.2

• 3rd Party restore

• Questions and Answers

2 |
 Domino 12 Backup
Introduction & Functionality

3 |
 Domino Backup Requirements
• Domino NSF files need to be backup-ed on-line
− Open file backup is not an option and leads to inconsistent databases!
− You either need a Domino aware backup software
− Or shutdown your Domino server for backup!

• VSS (Volume Shadow Copy) or file-system snapshots on Linux alone are not supported
− A Domino Backup integration needs to support Snapshot backup

• Customers ask for archive style transaction logging to allow point in time restores

• New requirements like backup Docker containers and also new backup vendors
− For example Veeam leveraging snapshot backup

4 |
 Domino V12 Design Goal
• Full featured Domino Backup integration to bridge between Domino and backup solutions

• Allow integration with any type of backup vendor via custom scripts

• Allow Domino to integrate with customer's favorite backup solution

• Support snapshot backup applications

• Allows broad functionality on the Domino backup/restore side

• Notes/Domino style UI in combination with favorite backup solution

5 |
 Main Functionality
• Flexible restore options
− Bring database online, point in time recovery,
− Disable replication, change replica ID, change title, disable all agents, etc.

• Command-Line/REST API, etc. integration for backup tools with file and snapshot backup

• Support for Win64 and Linux64 only

− AIX and OS400 have strong IBM vendor support already

• Recover documents and folders into original database

• Integrated DAOS restore missing NLOs leveraging an existing file-backup

− Focus is NSF & Translog backup. DAOS is a single file backup handled by any backup application

6 |
 Technical Background
• Problem
− Domino is constantly accessing files on disk; thus, file copy operations User
are causing inconsistent backups.

• Solution Domino
− Domino Backup API brings database into consistent state Backup API
− Now NSF/NTFs can be backed up on file-level
DELTA
− Write access is still available to end users. NSF

• Changes (aka deltas) are captured and must be applied to the Local Disk
backup to bring the database into consistent state!
− Delta files are merged during backup in case of file backup operations
− Or on restore in case of other operations
− The Domino backup & restore takes care of tracking delta files

7 |
 Backup and Restore Application Components
• backup – Backup servertask invoked via program document

• restore – Restore servertask either

− a.) invoked on command line or
− b.) running permanently monitoring the restore job database via -g option

• dominobackup.nsf
− Configuration
− Database Inventory and Backup Logs
− Restore interface and restore jobs

8 |
 dominobackup.nsf
• Configuration
− Per server or global configuration per platform
− Global settings

• Per Database Backup log repository

− Showing all backups per database and base for restore operations

• Per Backup log for NSF files and Translog

• Restore requests
− Derived from per database backup, when you select a restore

9 |
 Backup Concept : Full Backup
• Full backup only
− Allows to recover data at the point in time when the backup was taken

• With circular Transaction logging is enabled in Domino

− Limited point in time recovery if transaction logs are still on disk

• Typical setup:
− Daily Full backup
− Backup job running outside of business hours

Full Full Full Full Full Full

10 |
 Backup Scenario: Disk / File Share
• Configured out of the box
− Just needs a “disk” with sufficient space to backup databases
− Very simple to setup

• Databases changes during backup are automatically merged into the database backup!
− Consistent backup without the need to use the restore servertask
− Databases can be just copied back – Don’t overwrite existing databases on OS level when the server is up!!

• Any type of file-share

− Recommended: Storage supporting compression and de-duplication
▫ Cohesity → Backup share into backup repository (called “View”)
▫ e.g TrueNAS → https://www.truenas.com/ - leverages ZFS storage like many others on the market
▫ Other deduplicating storage like NetApp etc..

11 |
 Backup
Quick Start Backup Backup.nsf Domino
Task

• 1. load backup → creates new dominobackup.nsf Storage Storage

Local Disk
• Review configuration
− One pre-defined file copy configuration per platform
− Already enabled
− Check if the target location works in your environment
− Server needs write permissions to create the directory!

• 2. load backup → to start your first backup

12 |
 Restore Operations
• 1. Find the database to restore

• 2. Create a “restore job” using the action button

• 3. Specify the restore time and restore options

− Most important options are set by default

• 4. Submit the restore using action button

• 5. load restore
− Tip Load restore -g
− Allows to keep the restore application loaded
− If you have remote console permissions:
→ Action button to start restore

13 |
 File Types involved
• *.nsf, *.ntf, *.box
− Original databases to be backed up

• *.txn
− Transaction log files to be backed up

• *.DELTA
− Deleta files created during backup – if database changes during backup (e.g. names.nsf.DELTA)

• *.DAD
− Restored database before “activation” (e.g names.nsf.DAD)
− Restored with a temporary name to avoid server will find the database before it it recovered

14 |
 Restore operation in detail
Backup.nsf Domino Restore Task
• Servertask copies back database to restore location

• File name will have a .DAD extension to ensure the server Storage
Storage
does not access it meanwhile

• Disables replication and sets other settings Restore/

user.nsf

Local Disk
• Applies .DELTA file if available and recovers the database

• Renames the database to remove the .DAD extension

• Recovery documents and folders into original database if requested

15 |
 Restore documents & folders to original database

• Restore documents and folders

− Restore deleted documents & folders into original database!
− Finds deletion stubs and updates it to replace the deletion stub
− Folder notes are updated and replicated back to the original
database overwriting the deletion stub

Backup.nsf Domino Restore Task

Storage Storage

Mail/ Restore/
user.nsf user.nsf

Local Disk

16 |
 Backup Integrations

17 |
 Integration Points
• File backup commands – build-in, leveraging OS commands
− Most easy to setup backup/restore
− Customizable via @formulas
− Can be also leveraged for commercial software like Cohesity with their backup repository
− Used for backup, restore and backup prune operations (delete)

• Command-Line interface
− Very flexible, standardized interface for Windows (batch) and Linux (shell script)
− Customizable via @formulas
− Return strings can be passed to the backup software for backup reference and status

• Agent interface
− Can be for example used for REST requests leveraging HTTP Request class

18 |
 Backup Scenario: Integration on Script level
• Command Line backup with a backup application
− For example: Linux Borg Backup → https://borgbackup.readthedocs.io/en/stable/
− Or command line integration with a client like AWS S3 CLI or any other custom script

• Flow
− Domino backup servertask takes one database after another into backup mode
− OS level command is called to backup the database
− Bring database into normal operations mode
− Check if changes occurred during backup and create a delta file with changes

• Challenges
− Restore operation is required to bring a restored database online
− Deltas need to be applied back on restore

19 |
 HCL GitHub Project for Backup Integrations
• https://github.com/HCL-TECH-SOFTWARE/domino-backup

• https://opensource.hcltechsw.com/domino-backup/

20 |
 “Delta Files” created during backup
• The Domino Backup API brings databases into a consistent state for backup

• Databases can be backed up on file-level once backup is started

• Important: Changes in database are recorded and need to be applied to the backup database
to bring the database into consistent state!

• Delta files can be

− a.) Merged during backup in case of file backup operations
− b.) On restore in case of other operations

• Delta Files have to be merged in any case!

− But don't worry - The Domino backup & restore takes care about tracking delta files

21 |
 Implementation reference: S3 Storage Minio
• Command line integration

• Simple integration based on formulas not a “Cmd file”

− Good example to show how @Formula integration works
− Results are captured by the backup/restore task
▫ OK and Error strings can be used to check operation status
− Needs helper binary and config from S3 vendor (e.g. mc from Minio)

22 |
 Implementation reference: S3 Storage AWS CLI S3
• Same type of integration

• Just a different command line used

− In this case AWS CLI with S3 command

• Needs target specific configuration

− AWS credentials and target information for the bucket used

23 |
 HCL GitHub Project for Backup Integration Solutions

• Open Source repository for 3rd party backup integrations based on the framework Domino provides

• Integration point/operations documentation

• Additional information
− Technologies like snapshot
− Best practices around backup and storage optimization
− File system back-end operations

• Main entry point for supporting backup integrations

− It is not intended that HCL support should help to integrate applications

• Collaboration in the community via GitHub repository

− Backup vendors, customers, partners and HCL

24 |
 Snapshot Backup
Background & New Features

25 |
 Backup Scenario: Snapshot
• Works similar like previous scenario but brings all databases into backup mode at once

• Flow
− Bring all databases into backup mode
− Call an OS level command to take snapshot
− Bring all databases into normal operations mode
− Create a delta file with changes for each database with changes
− Operations on Domino side finished, snapshot and delta files are static and any file backup can be used

• Backup using snapshot

− Many different options depending on the solution used
− Most simple approach on Windows – without a backup application supporting snapshots itself
▫ Call a VSS snapshot command and to get a temporary snapshot
▫ Use any file backup solution to backup the snapshot along with delta files
26 |
 Backup Concept : Snapshot
• Flow:
1. Backup application brings all databases into backup mode
2. Snapshot is initiated – should not take more than a couple of sections
3. Delta files from backup are stored separate from the snapshot
4. Snapshot and delta files are backed up

• On restore usually the snapshot is mounted to “copy” databases back

• Delta files are usually saved to different backend or separate snapshot of another disk

Full Full Full Full Full

SNAP SNAP SNAP SNAP SNAP

SHOT SHOT SHOT SHOT SHOT

27 |
 VM Level Snapshots
• Many backup vendors support VM level/storage level snapshots

• Main technology used

− VMware VSphere snapshots leveraging their storage API
− Hyper-V snapshots
− Storage snapshots e.g. NetApp and others

• Already supported: Veeam backup leveraging VMware VSphere & Hyper-V backup
− Flow:
▫ 1. Veeam initiates backup to bring Domino into snapshot mode “Freeze” event
▫ 2. Veeam generates snapshot and unfreezes databases with “Post Thaw” event
▫ Implemented leveraging integration scripts for Windows and Linux to control “Domino Backup”

28 |
 Linux Snapshot Support
• No build-in snapshot functionality into Linux core
− But available for certain file systems

• OpenZFS
− Widely available for most distributions and supported by Domino
− OpenSource project forked from Sun long time ago and very mature enterprise grade file-system

• Btrfs
− Mainly used by SUSE and supported by Domino

• Both file-systems support snapshots created and managed by OS level commands

• Domino Backup invokes snapshot commands and manages snapshots

− Create, Mount, Delete
29 |
 What’s new in
Domino 12.0.2 Backup
Native Windows VSS Writer Implementation

30 |
 What’s next?
• The next logical step would be to make Domino a full native VSS Writer

31 |
 VSS Writer Support
• Volume Shadow Copy Service (VSS) supports application integrated snapshot operations

• “VSS Writer” allows to make an application fully “snapshot aware”

without any direct backup application integration or scripting

• Requires Domino to become a “VSS Writer” supported application

• Flow
− Domino registers as a “VSS Writer” using a Microsoft VSS API
− Backup application starts VSS Snapshot
− Windows sends event to all VSS Writers registered to “Freeze” their application
− Windows takes VSS snapshot
− Windows sends VSS “Post Thaw / Post Snapshot” event to application
− Domino processed delta data accumulated during snapshot operations (60 < sec)

32 |
 Domino VSS Writer Requirements
• Domino needs to register as a native VSS Writer on startup
− Leveraging the Microsoft VSS API

• Domino needs to permanently listen to VSS call-backs and handle

“Freeze” and “Post Thaw/Post Snapshot” events to initiate Domino Backup Snapshot operations

• Domino needs to be permanently registered as a VSS writer and respond to VSS events

• Works with all VSS backup aware backup applications without any additional scripting for backup!

• VSS writer integration is designed as a backup only integration

− Restore still requires an integration to “mount” snapshots to restore NSF files
− This is in alignment with backup vendor implementations → VSS is just used as a helper technology

33 |
 Domino VSS Writer
− VSS Admin Windows tool shows all registered VSS Writers
− Once registered, Domino is listed as a backup writer

34 |
 VSS Writer “AutoRecover” Support
• The biggest challenge for snapshot backup
− Snapshots cannot be modified
− Delta changes need to be stored separately and need to be applied
to the database on restore to make the NSF file consistent

• Solution
− VSS Writers VSS_VOLSNAP_ATTR_AUTORECOVER Option
− Allows a VSS Writer to update the snapshot in the OnPostSnapshot event to
▫ Merge delta information occurred during backup
▫ Make the database consistent for recovery without Domino restore operations
− Tricky but small modification in the snapshot flow in backup servertask to apply
changes directly into the writable snapshot in OnPostSnapshot event

35 |
 Domino Backup VSS Writer Flow
e notesdata
ackup application VSS backend Do ino ackupVSS Do ino ackup NSF files Snapshot
Admin

ackup
Start backup

Run backup

OnIdentify()

Signal interest in snapshot

OnFreeze()

Start snapshot backup

NSFBackupStart()

All DBs in backup mode

OnFreeze() finished

Snapshot

Create snapshot

OnPostSnapshot()

Mount writable

Stop backup

NSFBackupStop()

NSFBackupStartApplyChangeInfo()

NSF backup finished

OnPostSnapshot() finished

Snapshot created

Backup snapshot

Delete snapshot

Admin
ackup application VSS backend Do ino ackupVSS Do ino ackup e notesdata Snapshot
NSF files
36 |
 Implementation
• Separate “backupvss” servertask registering as a VSS backup writer

• Invokes Domino Backup servertask (backup -vss) to leverage “Domino Backup Snapshot Mode”

• In Freeze event waits for backup task to bring all databases into backup mode

• Fully functional implementation of VSS API

− Integrate VSS Writer functionality into Core Domino
− Separate task is needed to control backup servertask
− “backupvss” task is required to be permanently loaded to allow VSS to communicate with Domino

37 |
 VSS Writer Implementation Limitations
• NSF Data is required to be on a single volume for snapshot
− No support for external directory or NSF links pointing to a different volume
− No support for Windows junctions and comparable mount options
− Support for directory and NSF link on the same physical volume

• VSS Snapshot backup application is required to support “AutoRecovery” mode for full functionality
− Fallback to write delta files is still possible – In the same way it is supported in 12.0.1 today

• Restore integration still requires separate integration similar to Veeam integration available today
− Restore integration scripts are posted in GitHub repository
− No support for VSS restore operations
− Vendors backup to their own repository and have no direct VSS restore integration

• Only one Domino partition per Windows machine can be backed up via VSS

38 |
 Veeam Backup & Replication Domino Restore
Veeam Backup & Replication
on Windows 2022
Domino on
Windows

3 SSH
2 OpenSSH Server Configuration (JSON)

Domino + Restore Restore Script Private key 5

1 7 Copy

PowerShell Veeam
Backup & Replication
dominobackup.nsf VM Disks

6
Veeam Data Store
Mount

Backup
(Restore Point)

39 |
 Updated Veeam Integration
• Updated (and simplified) documentation for Veeam integration
− https://opensource.hcltechsw.com/domino-backup/backup-providers/veeam/install_vss_writer/

40 |
 Build your own Lab Environment
• Free Backup for up to 10 Workloads!
− Provides even free agents for workstations / notebooks
− Can be used in production!

• Full featured functionality for virtual environments

− VMware vSphere
− Microsoft Hyper-V
− https://www.veeam.com/backup-replication-system-requirements.html

• Community edition: https://www.veeam.com/virtual-machine-backup-solution-free.html

41 |
 Domino 12.0.1
New Features & Improvements

42 |
 Domino V12.0.1 End to End Logging
• In Domino 12.0 logs for command-line/script integrations are only parsed for errors
− For debugging you had to write the logs into files and manage them on your own

• Domino 12.0.1 backup automatically collects all standard output from invoked scripts/ commands
− When backup/restore scripts/commands fail, the output is stored in a temporary file
− Once the backup or restore operations completes, the full logfile is appended to the backup/restore note
− On by default and cannot be disabled, Option: log all output for debugging purposes (Main configuration tab)

− Redirect error output for scripts by adding 2>&1 direct to the command configuration
− Important: Redirect STDIN via </dev/null Linux and </dev/NUL on Windows

43 |
 Domino V12.0.1 Multi Restore Improvements
• In Domino V12.0 the multi restore operation was limited
− All databases had to come from the same backup
− There have been some UI issues not showing the right error messages

• Domino 12.0.1 now fully supports multi database restore

− Always the latest backups for a database matching the restore time, are used
− Databases don’t need to be in the same backup (e.g. incremental backups)

• How does it work?

1. Admin selects the restore time
2. Submit logic checks each selected database for matching backup and writes the time into each doc
3. Restore operation will restore each database from the specified backup date

44 |
 Domino V12.0.1 Multi Restore Optimization
• Challenge
− Restoring multiple database from a snapshot would potentially involve many mount/unmount operations
− For many backup vendors mounting a snapshot is a quite time consuming operation

• Solution
− New pre-restore and post-restore scripts to allow restore operation optimization

• Example logic:
− Restore DB operation script → Checks if the right snapshot is mounted
− → If yes, just copy the database from snapshot
− → If not mount the snapshot and copy the file
− Next file might hit the already mounted snapshot – or mounts another snapshot
− Post Restore Script → Unmounts all mounted snapshots

45 |
 Domino V12.0.1 Selective Backup Prune
• In Domino 12.0 backups are only pruned by retention time

• Domino 12.0.1 introduces a selected prune operation directly from the Log views
− Just select one or more backups to be pruned and run “load backup -p” ( or Run Prune Server Cmd)

46 |
 3rd Party Restore (Domino 12.0.1)
In case we still have time..
Slides have most of the details

47 |
 Domino 12.0.1 3rd Party Restore
• Backup vendors with full Domino support leveraging the standard C-API interface can now leverage
the Domino restore interface for restore operations

• Restore 3rd party restore mode

− Allows to integrate with your favorite Domino aware backup solution
− Leverages same type of integration used for the existing backup integrations

• Flow:
− Specify database to restore & run restore operation
− “restore” task executes integration script to trigger 3rd party restore operation
− 3rd party restores database and brings it online
− Domino restore performs additional restore operations (changing replica-ID, restoring documents & folders!)
− Existing DAOS restore operations can be combined

48 |
 Configure Domino 3rd Party Restore
• Same type of integration you know from full
integration operations

• Restore operation triggers the external backup

vendor for a full restore including bringing the
database online

• Domino restore operations will be executed

immediately after restore completes

• Tip:
− Restore time cannot be passed via Cmd operation
− Best choice is probably “Cmd Formula”
− Allows to pass any field in restore document

49 |
 Configure Domino 3rd Party Restore
• Tip, if you only want to change the date, there is a 3rd party date formula

− X:=@If (RestoreDateTime=""; @Now;RestoreDateTime);

Y:=@Text(@Year(X));M:=@Text(@Month(X));D:=@Text(@Day(X));H:=@Text(@Hour(X));N:=@Text(@Min
ute(X));S:=@Text(@Second(X));
@Repeat("0";4-@Length(Y))+Y + @Repeat("0";2-@Length(M))+M + @Repeat("0";2-@Length(D))+D +
@Repeat("0";2-@Length(H))+H + @Repeat("0";2-@Length(N))+N + @Repeat("0";2-@Length(S))+S;

50 |
 Enable 3rd Party Restore Operations
• Open global configuration

• Enable “Restore 3rd party”

• Brings dominobackup.nsf into 3rd party operations mode

51 |
 Perform 3rd Party Restore
• 3rd party restore can obviously not use the database inventory

• Restore operations are triggered by entering the database name manually

52 |
 Tip: 3rd Party Restore → Select Databases via SmartIcon

• SmartIcons are globally available in any database

• Idea: Run SmartIcon on person/mail-in or catalog.nsf document to create a restore document

Server := @Subset(@DbName; 1);

BackupDB := "dominobackup.nsf";
@If (Server = ""; @Prompt([Ok];"Error"; "Cannot run on local database!"); "");
@If (Server = ""; @Return (""); "");
Database := @If (Mailfile != "";
@If (@Ends(@LowerCase(MailFile);".nsf");MailFile;MailFile+".nsf"); PathName !="";PathName;"");
@Command([Compose]; Server : BackupDB; "RestoreJob");
@UpdateFormulaContext;
@SetField ("ServerName"; Server);
@SetField ("PathnameRestore"; Database);
@Command([EditGotoField]; "RestorePath");
53 |
 Q&A

54 |
 Domino Storage Optimization

55 |
 Domino Storage Optimization before Backup!
• Standard storage is cheap
− I/O intensive storage is more expensive
− High available storage can be more expensive as well
− Storage you need to backup every night can be expensive for your company over time!
− The night has usually only ~12 hours for backup..

• Domino provides many storage optimization options

− Most of them available since Domino 8.5.x!

• You have to revisit your current database maintenance operations!

− Database maintenance should usually scheduled before the backup
− You don’t need to compact nor fixup databases every night!

• Domino storage optimization and database maintenance operations have to be addressed

before you can optimize your backup!
56 |
 NSF Database Size before Optimization
NSF Storage
Design 4%
NIF 13%
Documents 10%

NIF
Attachments
Documents
Design

Attachments 73%

57 |
 Design Note Compression (Domino 8.0)
• Design is stored compressed on disk
− Only internal storage format changes → No difference visible to applications even on API layer
− Opening the design notes automatically uncompressed the design
▫ Tip: Enable network compression to reduce network I/O for remote operations

• Introduced in ODS 48 and enabled for Domino templates by default

• Reduces design of database design by 40 – 50%

• Just marginal storage reduction on it’s own → but also lowers I/O for design by 50%

• Example Mail 12.0.1 template:

− Uncompressed 37 MB
− Compressed (default): 17 MB
58 |
 Document Compression (Domino 8.0.1)
• Works in the same way as design compression just for documents
− All items including richtext
− Only internal storage format changed. Also transparent to all APIs
− Network compression helps for remote transactions

• Reduces the size of a document by 40-50%

− Reduces storage and reduces I/O load!
− Needs less backup space & time every night!

• Enabling design and document compression

− Load compact -n -v -* → enables database properties
▫ Compact is required to re-compress existing design/data
− load DBMT –ct 4

59 |
 Design and Document compression I/O Reduction

• Main focus is I/O reduction

− CPU utilization stays the same
− Amount of I/O and even more important: IOPS (I/O per second) is reduced by 20%

• Data from a Lotusphere 2008 presentation

60 |
 Attachment Compression
• Compresses attachments with LZ1 automatically

• Only internal representation of attachment changes

− Transparent to application

• Compression rate is checked before storing compressed

− Helpful when attachment is already compressed
− Existing compress formats including image compression are not compressed

61 |
 DAOS – Domino Attachment and Object Service
• Available since Domino 8.5

• Moves attachments to a
separate file system with
one *.NLO file per
attachment

• Deduplicates attachments

• Reduces the NSF files

up to 70%!

• Historic graphics from 8.5

used on purpose ;-)

62 |
 DAOS Benefits
• Reduce disk space depending on degree of content overlap
− Usually 30-50% storage reduction by deduplication
− NSF file is reduced up to 70%

• Improved performance
− Attachments are written once per server
− Less IOPS, with larger block size

• Large objects are stored outside the NSF in static files

− Improve compact performance (less data stored + no large objects moved)
− Reduce file-system fragmentation
− Large reduction in incremental backup costs
− DAOS repository isolates large blocks of data into separate, unchanging files

•63 Clients
|
check if DAOS object is already on server before sending!
 Mail Database Size with DAOS
• 40-50% total space reduction by
deduplication Mail-Size 12 Month measured -2 / +2 Years
5000
• Reduction of NSF by 60 – 70% 4500

4000
• Almost linear growth for NSF size 3500
with DAOS enabled 3000 Total
2500 DAOS
NSF
2000

1500

1000

500

0
-2,0 -1,5 -1 year -0,5 Now +0,5 year +1 year +1,5 +2 years
years years years years

64 |
 Enable DAOS
• Configure DAOS store on separate disk (server document)
− On Linux and AIX it should be a sub directory of a mount point

• Specify DAOS threshold: 128 KB - 512 KB

− Usually 256 K is a good starting point

• Disable DAOS encryption

− before Domino 12.x via Notes.ini: DAOS_ENCRYPT_NLO=0 • Load compact –DAOS ON mail/

• Lower number of files per sub directory from 40000 to 10000 to enable DAOS settings
− Notes:ini DAOS_MAX_FILES_PER_SUBCONTAINER=10000
• Load DBMT mail/
• Wait for settings to be stored in notes.ini and restart server
to move attachments to DAOS

65 |
 NIFNSF
• NIF = Notes Index Facility

• View/Folder index data is stored in NSF file by default

• NIFNSF allows to store NIF data in a separate file

− Available since Domino 9.0.1 FP8

• For mail databases NIF can be up to 10% of the total space used
− For applications the index can be even bigger

• Only NSF files require backup. New *.NDX files only contain index data

• Separate file allows optimized locking of file

− Important for runtime and for indexing!

66 |
 Configure NIFNSF
• NIFNSFENABLE=1
− Enables NIFNSF in general
− By default NDX file is stored next to NSF file

• NIFBASEPATH=/local/nif (Optional)
− Use different base path outside the data directory for NIF data

• CREATE_NIFNSF_DATABASES=1 (Optional)
− Create new databases with NIFNSF enabled

• Enable per database:

− Load compact –NIFNSF ON

• Move existing index data to NDX file

− Load DBMT
67 |
 Move FT Index to different disk
• Specially for snapshot backups, moving FT to a different disk/volume is important

• Notes.ini setting to define a directory outside notesdata directory.

• Example on Linux: FTBASEPATH=/local/ft

• Allows to store FT index into any location outside data directory → separate disk
− It should be a local disk, not a remote mount!
− Same I/O requirements than NSF

68 |
 DBMT replaces Compact & Fixup
• Available since Domino 8.5, but still not used by many admins

• Allows fixup, updall and compact operations in one task

• Multi threaded and can run multiple compacts in parallel

• Copy style backup with disk space pre-allocation when creating a new NSF
− New database with low number of fragments
− Notes.ini DBMT_PREFORMAT_PERCENT=n to specify how much space is pre-allocated
− DAOS aware calculating the new size in advance – e.g. DBMT_PREFORMAT_PERCENT=110

• Improvement of maintenance operations for

− Weekend compact operations
− Daily NIF and NSF operations

69 |
 DBMT Example
• During the week only maintain indexes and database purge etc
− No compact, 4 FT and 4 NIF update threads in parallel. Rebuild FT index every 30 days
dbmt -ut 4 -ft 4 -ct 0

• Saturday before backup

− Compact 4 threads only if database has more than 60 MB free space for max 3 hours
− then compact all databases not compacted for 21 days for max 3 hours
dbmt -ut 0 -ft 4 -ct 4 -tl 180 -ncl -S 60M –cnd 1
dbmt -ut 0 -ft 4 -ct 4 -tl 180 -ncl -cnd 21

• Sunday
− Update NIF index and FT index with 4 threads each and rebuild FT Index if 30 days old
dbmt -ut 4 -ft 4 -ct 0 -fnd 30
70 |
 Storage Optimization Summary
• Leverage DAOS!
− Reduces NSF files up to ~70%
− DAOS is a simple file-backup of static files!
− Recommended threshold: ~256 KB

• Use database design and document compression

− Saves ~50% of design/document data

• Enable NIFNSF to store index outside the NSF

− Reduces NSF size by ~10% of the original database

• Use DBMT to maintain your databases

− Don’t compact too often!

71 |
 Leverage Domino archiving
• Database size grows exponentially
Mail File Size 12 Month Archive
− Usually doubles every 12 to 18 month
12000

• Archiving lowers the exponential growth factor 10000

of your active databases!
8000

• Archiving splits databases into Long Term

Active
6000
− Smaller active often changing database
4000
− Larger static databases, with less I/O
requirements 2000

• Tip: If you archive once per month, you only 0

-2,0 -1,5 -1 -0,5 Now +0,5 +1 +1,5 +2,0
need to backup only once after archiving years years year years year year years years

• Domino server based archiving is policy

based and easy to setup
72 |