nipyapi 1.1.0

Features

Three layers of Python support for working with Apache NiFi:

• Top-level examples (see examples directory with usage guide)

• Mid-level Client SDK for typical complex tasks

• Low-level Client SDKs for the full API implementation of NiFi and selected sub-projects

Functionality Highlights:

• Profiles System: One-command environment switching with nipyapi.profiles.switch('single-user') (see profiles documentation)

• Modern Authentication: Built-in support for Basic Auth, mTLS, OIDC/OAuth2, and LDAP

• Environment Management: YAML/JSON configuration with environment variable overrides

• Detailed documentation of the full SDK at all levels

• CRUD wrappers for common task areas like Processor Groups, Processors, Clients, Buckets, Flows, etc.

• Convenience functions for inventory tasks, such as recursively retrieving the entire canvas, or a flat list of all Process Groups

• Support for scheduling and purging flows, controller services, and connections

• Support for import/export of Versioned Flows from various sources

• Integrated Docker workflow with Makefile automation and profile-based testing

Please see the issue register for more information on current development.

Quick Start

You need a running NiFi instance to connect to. Choose the approach that fits your situation:

Path A: Quick Start with Docker (Recommended for New Users)

Use our provided Docker environment for immediate testing:

# Clone the repository (includes Docker profiles and Makefile)
git clone https://github.com/Chaffelson/nipyapi.git
cd nipyapi

# Install NiPyAPI in development mode
pip install -e ".[dev]"

# Start complete NiFi environment (this may take a few minutes)
make certs && make up NIPYAPI_PROFILE=single-user && make wait-ready NIPYAPI_PROFILE=single-user

# Test the connection
python3 -c "
import nipyapi
nipyapi.profiles.switch('single-user')
version = nipyapi.system.get_nifi_version_info()
print(f'✓ Connected to NiFi {version}')
"

Path B: Connect to Your Existing NiFi

If you already have NiFi running, install and configure:

# Install NiPyAPI
pip install nipyapi

# Create your own profiles.yml
mkdir -p ~/.nipyapi
cat > ~/.nipyapi/profiles.yml << EOF
my-nifi:
  nifi_url: https://your-nifi-host.com/nifi-api
  registry_url: http://your-registry-host.com/nifi-registry-api
  nifi_user: your_username
  nifi_pass: your_password
  nifi_verify_ssl: true
EOF

# Test your custom profile
python3 -c "
import nipyapi
nipyapi.config.default_profiles_file = '~/.nipyapi/profiles.yml'
nipyapi.profiles.switch('my-nifi')
version = nipyapi.system.get_nifi_version_info()
print(f'✓ Connected to NiFi {version}')
"

Path C: Manual Configuration (Advanced)

For advanced use cases without profiles:

# Install NiPyAPI
pip install nipyapi

# Configure in Python code
import nipyapi
from nipyapi import config, utils

# Configure endpoints
config.nifi_config.host = 'https://your-nifi-host.com/nifi-api'
config.registry_config.host = 'http://your-registry-host.com/nifi-registry-api'

# Configure authentication
utils.set_endpoint(config.nifi_config.host, ssl=True, login=True,
                   username='your_username', password='your_password')

Next Steps: Start Using NiPyAPI

Once your environment is set up, you can start using NiPyAPI:

import nipyapi

# If using profiles (Paths A or B)
nipyapi.profiles.switch('single-user')  # or your custom profile name

# Basic operations
root_pg_id = nipyapi.canvas.get_root_pg_id()
version = nipyapi.system.get_nifi_version_info()
process_groups = nipyapi.canvas.list_all_process_groups()

print(f"Connected to NiFi {version}")
print(f"Root Process Group: {root_pg_id}")
print(f"Found {len(process_groups)} process groups")

Built-in Docker Profiles:

When using Path A (Docker), these profiles are available:

• single-user - HTTP Basic authentication (easiest to start with)

• secure-ldap - LDAP authentication over TLS

• secure-mtls - Mutual TLS certificate authentication

• secure-oidc - OpenID Connect (OAuth2) authentication

See docs/profiles.rst for complete profiles documentation and docs/migration.rst for upgrading from 0.x.

Examples and Advanced Usage:

• Flow Development Lifecycle: See examples/fdlc.py for multi-environment workflow patterns

• Interactive Sandbox: Run make sandbox NIPYAPI_PROFILE=single-user for experimentation (requires Docker setup)

• Custom Profiles: Create your own profiles.yml for production environments

• Environment Variables: Override any setting with NIFI_API_ENDPOINT, NIFI_USERNAME, etc.

• Testing Different Auth Methods: Use make up NIPYAPI_PROFILE=secure-ldap to try LDAP authentication

Please check out the Contribution Guide if you are interested in contributing to the feature set.

Background and Documentation

NiFi Version Support

Python Support