Final GSoC 2017 Report

This is a summary of what I did as GSoC participant working for the 389 Directory Server project.

Overview

The initial and main goal of this project was to develop command line tools for managing some of the existing Directory Server plugins in an easier way. This goal has been fulfilled and I additionally did some extra work on the lib389 python administration framework. Moreover, I experimented a bit with the C code base – the actual server – and worked on a few issues there as well.

What I did in sort:

  • Added dsconf support for managing the following plugins: MemberOf plugin, Referential Integrity plugin, USN plugin, Rootdn Access Control plugin.
  • Wrote code for executing server tasks through dsconf, such as Dynamic Schema Reload, MemberOf fixup task, etc.
  • Wrote command line tests and in some cases functional tests as well for testing the functionality of some plugins along with their command line behavior.
  • Worked on various other smaller issues on the lib389 codebase from fixing bugs and test cases, and adding new functionality to helping port things to python3.
  • Did some work on the 389-ds-base codebase for performing syntax checking on plugins and logging error messages/warnings when needed.

Examples

I don’t understand what you did exactly, can you show me examples? Sure. Until recently you had to apply LDIF files for configuring server plugins . You can see such an example of applying an LDIF in a previous post of mine: dsconf: Adding support for MemberOf plug-in. Or for checking a plugin’s status (whether it’s enabled or disabled) you had to make an LDAP search using some utility like the ldapsearch tool.

Let’s see a few cool things we can now do with the new command line toolkit.

If we want to examine the status of USN plugin it’s only a matter of:

~ $ dsconf localhost usn status 
USN is disabled

…then enable it because it is off:

~ $ dsconf localhost usn enable
Enabled USN

…and turn the global mode on:

~ $ dsconf localhost usn global on
USN global mode enabled

Let’s say I want to set the update the interval of Referential Integrity plugin:

~ $ dsconf localhost referint delay 60
referint-update-delay set to "60"

…and then see on which attributes referint is operating:

~ $ dsconf localhost referint attrs
referint-membership-attr: member
referint-membership-attr: uniquemember
referint-membership-attr: owner
referint-membership-attr: seeAlso

Now, let’s play with memberOf. Say I want to make it work on all backends:

~ $ dsconf localhost memberof allbackends on
memberOfAllBackends enabled successfully

…but I don’t want it to skip nested groups:

~ $ dsconf localhost memberof skipnested off
memberOfSkipNested unset successfully

I wonder if I can see the whole memberOf config entry at once. Of course!

~ $ dsconf localhost memberof show
dn: cn=MemberOf Plugin,cn=plugins,cn=config
cn: MemberOf Plugin
memberofallbackends: on
memberofattr: memberOf
memberofgroupattr: member
memberofgroupattr: uniqueMember
memberofskipnested: off
nsslapd-plugin-depends-on-type: database
nsslapd-pluginDescription: memberof plugin
nsslapd-pluginEnabled: on
nsslapd-pluginId: memberof
nsslapd-pluginInitfunc: memberof_postop_init
nsslapd-pluginPath: libmemberof-plugin
nsslapd-pluginType: betxnpostoperation
nsslapd-pluginVendor: 389 Project
nsslapd-pluginVersion: 1.3.7.1.20170811git63ea19dcb
objectClass: top
objectClass: nsSlapdPlugin
objectClass: extensibleObject

And what about running the MemberOf fixup task?

~ $ dsconf localhost memberof fixup -b "dc=example,dc=com"  
Attempting to add task entry... This will fail if MemberOf plug-in is not enabled.
Successfully added task entry cn=memberOf_fixup_2017_08_23_18_49_50,cn=memberOf task,cn=tasks,cn=config

How about interacting with Rootdn Access Control plugin now in order to configure IP based access control for the Directory Manager?

~ $ dsconf localhost rootdn ip allow 127.0.0.1
127.0.0.1 added to rootdn-allow-ip
~ $ dsconf localhost rootdn ip allow 127.0.0.2
127.0.0.2 added to rootdn-allow-ip
~ $ dsconf localhost rootdn ip deny "192.168.1.*"
192.168.1.* added to rootdn-deny-ip

And if I actually want to inspect what I did:

~ $ dsconf localhost rootdn ip 
rootdn-allow-ip: 127.0.0.1
rootdn-allow-ip: 127.0.0.2

rootdn-deny-ip: 192.168.1.*

These are just a few examples and by using the new command line toolkit you can actually do every operation that is possible by applying LDIF files or doing LDAP searches, only quicker and easier now.

Benefits

So why would I prefer to use dsconf to configure the server instead of doing it the “standard” way? Well, for starters because it’s obviously easier. Applying LDIFs for performing the simplest operations can soon become very tedious.

But there’s actually an even bigger advantage. The API now becomes _discoverable_. The admin now doesn’t need to go to the docs to see what options are there for each plugin (and you do know that most admins are lazy, right?).

For example, if they want to know what they can do with the USN plugin they can type:

~ $ dsconf localhost usn -h 
usage: dsconf instance usn [-h]
 {show,enable,disable,status,global,cleanup} ...

positional arguments:
 {show,enable,disable,status,global,cleanup}
                      action
    show              display plugin configuration
    enable            enable plugin
    disable           disable plugin
    status            display plugin status
    global            get or manage global usn mode
    cleanup           run the USN tombstone cleanup task

Or if they wants to see what options there are for configuring IP based access control for the Directory Manger they can run:

~ $ dsconf localhost rootdn ip -h 
usage: dsconf instance rootdn ip [-h] {allow,deny,clear} ...

positional arguments:
 {allow,deny,clear}   action
    allow             allow IP addr or IP addr range
    deny              deny IP addr or IP addr range
    clear             reset IP-based access policy

We additionally wish to add extensive help for dsconf in the future and better command line documentation of what each attribute does and how it can be configured. The ultimate goal is to make dsconf and friends a complete one stop for admins when configuring the server.

Code

Enough talk! Show me the code!

All lib389 commits: https://goo.gl/WWAHmH
All 389-ds-base commits: https://goo.gl/eSaS1w

Although numbers do not mean a lot in that case, according to git 3,337 LOC have been added to the lib389 code base and 132 LOC have been removed in a total of 19 commits that have been merged.

Bugs Found

While working on the project I have discovered a few bugs (and fixed some of them).

Here are some examples:

  • #49284 – DS crashes when trying to completely remove some optional memberOf attributes
  • #49309 – Referential Integrity does not perform syntax checking on referint-update-delay
  • #49341 – dbscan is broken for entryrdn
  • #49224 – prefixdir in config.log set to ‘NONE’
  • #49274 – Memberof autoaddoc attribute shall accept only specific object classes

Future Work

There are many more interesting challenges to be faced, and that’s why I plan to continue contributing to the project whenever I can and whenever I find interesting tickets to work on.

I have already started working on an issue about performing a database dump on emergency recovery scenarios that will be completed outside the GSoC period. I’m looking forward to learning new things and facing new challenges!

Acknowledgments

I can’t emphasize enough how grateful I am for having the chance to have William Brown as a mentor. I learned so much during this period and William constantly motivated me to do my best for the project and the community. I would also like to thank the DS team, and the Fedora Project in general for this great experience and collaboration.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s