LDAP

Module ldap

This plugin extracts the following:

• People
• Names, emails, titles, and manager information for each person
• List of groups

CLI based Ingestion

Install the Plugin

pip install 'acryl-datahub[ldap]'

Starter Recipe

Check out the following recipe to get started with ingestion! See below for full configuration options.

For general pointers on writing and running a recipe, see our main recipe guide.

source:
  type: "ldap"
  config:
    # Coordinates
    ldap_server: ldap://localhost

    # Credentials
    ldap_user: "cn=admin,dc=example,dc=org"
    ldap_password: "admin"

    # Options
    base_dn: "dc=example,dc=org"

sink:
  # sink configs

Config Details

Options

Note that a . is used to denote nested fields in the YAML recipe.

View All Configuration Options

Field [Required]	Type	Description	Default	Notes
attrs_list [✅]	array(string)		None
base_dn [✅]	string	LDAP DN.	None
custom_props_list [✅]	array(string)		None
drop_missing_first_last_name [✅]	boolean	If set to true, any users without first and last names will be dropped.	True
filter [✅]	string	LDAP extractor filter.	(objectClass=*)
group_attrs_map [✅]	object		None
ldap_password [✅]	string	LDAP password.	None
ldap_server [✅]	string	LDAP server URL.	None
ldap_user [✅]	string	LDAP user.	None
page_size [✅]	integer	Size of each page to fetch when extracting metadata.	20
platform_instance [✅]	string	The instance of the platform that all assets produced by this recipe belong to	None
user_attrs_map [✅]	object		None
env [✅]	string	The environment that all assets produced by this connector belong to	PROD
stateful_ingestion [✅]	StatefulStaleMetadataRemovalConfig	Base specialized config for Stateful Ingestion with stale metadata removal capability.	None
stateful_ingestion.enabled [❓ (required if stateful_ingestion is set)]	boolean	The type of the ingestion state provider registered with datahub.	None
stateful_ingestion.ignore_new_state [❓ (required if stateful_ingestion is set)]	boolean	If set to True, ignores the current checkpoint state.	None
stateful_ingestion.ignore_old_state [❓ (required if stateful_ingestion is set)]	boolean	If set to True, ignores the previous checkpoint state.	None
stateful_ingestion.remove_stale_metadata [❓ (required if stateful_ingestion is set)]	boolean	Soft-deletes the entities present in the last successful run but missing in the current run with stateful_ingestion enabled.	True

Schema

The JSONSchema for this configuration is inlined below.

{
  "title": "LDAPSourceConfig",
  "description": "Config used by the LDAP Source.",
  "type": "object",
  "properties": {
    "env": {
      "title": "Env",
      "description": "The environment that all assets produced by this connector belong to",
      "default": "PROD",
      "type": "string"
    },
    "platform_instance": {
      "title": "Platform Instance",
      "description": "The instance of the platform that all assets produced by this recipe belong to",
      "type": "string"
    },
    "stateful_ingestion": {
      "$ref": "#/definitions/StatefulStaleMetadataRemovalConfig"
    },
    "ldap_server": {
      "title": "Ldap Server",
      "description": "LDAP server URL.",
      "type": "string"
    },
    "ldap_user": {
      "title": "Ldap User",
      "description": "LDAP user.",
      "type": "string"
    },
    "ldap_password": {
      "title": "Ldap Password",
      "description": "LDAP password.",
      "type": "string"
    },
    "base_dn": {
      "title": "Base Dn",
      "description": "LDAP DN.",
      "type": "string"
    },
    "filter": {
      "title": "Filter",
      "description": "LDAP extractor filter.",
      "default": "(objectClass=*)",
      "type": "string"
    },
    "attrs_list": {
      "title": "Attrs List",
      "description": "Retrieved attributes list",
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "custom_props_list": {
      "title": "Custom Props List",
      "description": "A list of custom attributes to extract from the LDAP provider.",
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "drop_missing_first_last_name": {
      "title": "Drop Missing First Last Name",
      "description": "If set to true, any users without first and last names will be dropped.",
      "default": true,
      "type": "boolean"
    },
    "page_size": {
      "title": "Page Size",
      "description": "Size of each page to fetch when extracting metadata.",
      "default": 20,
      "type": "integer"
    },
    "user_attrs_map": {
      "title": "User Attrs Map",
      "default": {},
      "type": "object"
    },
    "group_attrs_map": {
      "title": "Group Attrs Map",
      "default": {},
      "type": "object"
    }
  },
  "required": [
    "ldap_server",
    "ldap_user",
    "ldap_password",
    "base_dn"
  ],
  "additionalProperties": false,
  "definitions": {
    "DynamicTypedStateProviderConfig": {
      "title": "DynamicTypedStateProviderConfig",
      "type": "object",
      "properties": {
        "type": {
          "title": "Type",
          "description": "The type of the state provider to use. For DataHub use `datahub`",
          "type": "string"
        },
        "config": {
          "title": "Config",
          "description": "The configuration required for initializing the state provider. Default: The datahub_api config if set at pipeline level. Otherwise, the default DatahubClientConfig. See the defaults (https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/graph/client.py#L19)."
        }
      },
      "required": [
        "type"
      ],
      "additionalProperties": false
    },
    "StatefulStaleMetadataRemovalConfig": {
      "title": "StatefulStaleMetadataRemovalConfig",
      "description": "Base specialized config for Stateful Ingestion with stale metadata removal capability.",
      "type": "object",
      "properties": {
        "enabled": {
          "title": "Enabled",
          "description": "The type of the ingestion state provider registered with datahub.",
          "default": false,
          "type": "boolean"
        },
        "ignore_old_state": {
          "title": "Ignore Old State",
          "description": "If set to True, ignores the previous checkpoint state.",
          "default": false,
          "type": "boolean"
        },
        "ignore_new_state": {
          "title": "Ignore New State",
          "description": "If set to True, ignores the current checkpoint state.",
          "default": false,
          "type": "boolean"
        },
        "remove_stale_metadata": {
          "title": "Remove Stale Metadata",
          "description": "Soft-deletes the entities present in the last successful run but missing in the current run with stateful_ingestion enabled.",
          "default": true,
          "type": "boolean"
        }
      },
      "additionalProperties": false
    }
  }
}

Code Coordinates

• Class Name: datahub.ingestion.source.ldap.LDAPSource
• Browse on GitHub

Questions

If you've got any questions on configuring ingestion for LDAP, feel free to ping us on our Slack