What’s new in OpenDJ 3.0 – Part II

Yesterday, I’ve talked about the most important change in OpenDJ 3.0, that is the new PDB Backend. Let me detail other new and improved features of OpenDJ 3.0, still related to backends and replication.

As part of the work for the new backend, we’ve worked on the import process, in order to make it more I/O efficient and thus faster.

Here’s some numbers, importing 1 000 000 users in OpenDJ.

In OpenDJ 2.6.3:

$ import-ldif -l ../1M.ldif -n userRoot
[03/Feb/2016:15:41:42 +0100] category=RUNTIME_INFORMATION severity=NOTICE msgID=20381717 msg=Installation Directory: /Space/Tests/Blog/2.6/opendj
...
[03/Feb/2016:15:42:54 +0100] category=JEB severity=NOTICE msgID=8847454 msg=Processed 1000002 entries, imported 1000002, skipped 0, rejected 0 and migrated 0 in 71 seconds (average rate 13952.5/sec)

In OpenDJ 3.0, with the JE Backend:

$ import-ldif -l ../../1M.ldif -n userRoot
[03/02/2016:15:45:19 +0100] category=UTIL seq=0 severity=INFO msg=Installation Directory: /Space/Tests/Blog/3.0/opendj
...
[03/02/2016:15:46:22 +0100] category=PLUGGABLE seq=74 severity=INFO msg=Processed 1000002 entries, imported 1000002, skipped 0, rejected 0 and migrated 0 in 62 seconds (average rate 15961.2/sec)

In OpenDJ 3.0, with the PDB Backend

$ import-ldif -l ../../1M.ldif -n userRoot
[03/02/2016:15:59:38 +0100] category=UTIL seq=0 severity=INFO msg=Installation Directory: /Space/Tests/Blog/3.0/opendj
...
[03/02/2016:16:00:38 +0100] category=PLUGGABLE seq=48 severity=INFO msg=Processed 1000002 entries, imported 1000002, skipped 0, rejected 0 and migrated 0 in 58 seconds (average rate 17038.7/sec)

We’ve also completely reworked the storage layer for the replication changes, moving away from the BDB JE database. Instead, we’re using direct files, again providing much smaller disk occupancy (and thus optimising I/Os) but also allowing much more efficient purging of old data.

As part of these changes, we’ve made serious improvements to the way the replication changes can be read and searched using LDAP under the “cn=Changelog” suffix. More importantly, we’ve now have a way to ensure a complete ordering of the changes published, and thus consistency of their “changeNumbers”. That is to say that now, when reading “cn=Changelog” on different replicated servers, the change with “ChangeNumber=N” will be the same on all servers, allowing applications that read these changes to failover from one server to another. We’ve added a way to resynchronise these ChangeNumbers when adding a new replica to an existing topology, or when restoring one after a maintenance period.

Still on the subject of the ChangeLog, we’ve added another level of security to it, by introducing a “changelog-read” privilege that provides a better control on which applications and users are allowed to read the data from the “cn=Changelog” suffix.

That’s all for today. Tomorrow, I will continue with all the other new features and enhancements in OpenDJ 3.0.

If you have done it yet, you can download OpenDJ 3.0 from ForgeRock’s BackStage and start playing with it. And check the Release Notes for more information.