Skip to content
Aidin Documentation

Python SDK Overview

The Aidin Python SDK (aidin) is a high-level Python toolkit designed for authoring, compiling, and deploying machine learning pipelines on the Aidin AI Platform, targeting the managed KFP v2 (Kubeflow Pipelines v2) engine.

Version 0.4.0 introduces hardened authentication lifecycle controls, identity-scoped credential caching, and full compatibility with KFP v2.

Architecture

Section titled “Architecture”

The Aidin SDK is structured around four foundational layers:

+-------------------------------------------------------------+
|                      Developer Code                         |
|   @aidin.component             @aidin.pipeline              |
+-------------------------------------------------------------+
|                   Transparent DSL Layer                     |
|  - Injects configurable PyPI index (AIDIN_PIP_INDEX_URL)    |
|  - Injects default base image (COMPS_BASE_IMAGE)            |
|  - Re-exports KFP v2 types, artifacts, and control flow     |
+-------------------------------------------------------------+
|                  Compilation & Build Engine                 |
|  - Compiles Python DSL to KFP v2 YAML specifications        |
|  - Optional container build & push (harbor.aidin-ai.com)    |
|  - Automatic image reference rewriting in pipeline YAML     |
+-------------------------------------------------------------+
|               Authentication & Client Runtime               |
|  - Authentik -> Token Exchanger -> K8s SA TokenRequest flow |
|  - Secure client cache (~/.aidin/<project>/token.json 0600) |
|  - Clock-skew aware expiration validation (60s buffer)      |
|  - ProjectClient extending kfp.Client                       |
+-------------------------------------------------------------+

1. Transparent DSL (aidin.dsl)

Section titled “1. Transparent DSL (aidin.dsl)”

The SDK provides a transparent abstraction over kfp.dsl. Developers can import all standard KFP decorators, artifact types (Input, Output, Dataset, Model), metrics, and control flow operators (Condition, ParallelFor, ExitHandler) directly from aidin.dsl.

Aidin enhances standard components by automatically configuring:

  • Configurable PyPI index URL via AIDIN_PIP_INDEX_URL for packages specified in packages_to_install (defaults to environment PyPI without hardcoded external dependencies).
  • Default base container images resolved from the COMPS_BASE_IMAGE environment variable or defaulting to python:3.11.

2. Compilation and Packaging Engine (aidin.compile)

Section titled “2. Compilation and Packaging Engine (aidin.compile)”

The compilation engine converts Python pipeline functions into KFP v2 YAML packages. When project-level imports or custom dependencies are required across components, the --build flag builds a unified container image, pushes it to the Harbor OCI registry (harbor.aidin-ai.com/aidin-platform), and rewrites image references in the generated YAML package.

3. Authentication & Credential Lifecycle (aidin.shared.auth)

Section titled “3. Authentication & Credential Lifecycle (aidin.shared.auth)”

The SDK interacts with the upstream Authentik identity provider and the internal Aidin Token Exchanger. It exchanges an upstream user token for a short-lived, namespace-scoped Kubernetes ServiceAccount token. Upstream credentials are never written to disk, and cached service tokens are protected with strict POSIX permissions (0600).

4. Client Runtime (aidin.ProjectClient)

Section titled “4. Client Runtime (aidin.ProjectClient)”

The ProjectClient class extends kfp.Client, binding pipeline operations to a specific Aidin project namespace with verified credentials and automated token lifecycle checks.

Supported Python Versions

Section titled “Supported Python Versions”

The Aidin SDK strictly requires:

  • Python >= 3.9 (fully tested across Python 3.9, 3.10, and 3.11).
  • Python 3.14 is currently not supported by upstream kfp dependencies.

Installation

Section titled “Installation”

From Hamgit PyPI Package Registry

Section titled “From Hamgit PyPI Package Registry”

To install the official release package from Hamgit’s native PyPI package registry:

Terminal window
pip install aidin --index-url https://__token__:<your_token>@hamgit.ir/api/v4/projects/23708/packages/pypi/simple

If you manage dependencies using pip.conf or environment variables, configure the extra index URL:

[global]
extra-index-url = https://__token__:<your_token>@hamgit.ir/api/v4/projects/23708/packages/pypi/simple

OCI Execution Image (Harbor)

Section titled “OCI Execution Image (Harbor)”

A pre-built, non-root execution container image is published in Harbor for Kubernetes Jobs and automated execution:

Terminal window
docker pull harbor.aidin-ai.com/aidin-platform/aidin-sdk:0.4.0

Immutable digest reference:

harbor.aidin-ai.com/aidin-platform/aidin-sdk@sha256:28c605b3e05dc3c4b32d332bf9c8ab7873412206fa8b7aeb917b3e86c8928946

From Source

Section titled “From Source”

For internal development and testing:

Terminal window
git clone git@hamgit.ir:adin/mlops/sdk/new-sdk.git
cd new-sdk
pip install -e .

Verification

Section titled “Verification”

Verify your installation and confirm the SDK version:

Terminal window
aidin --version

Expected output:

aidin, version 0.4.0

Core Dependencies

Section titled “Core Dependencies”
Package Version Constraint Purpose
kfp >=2.4.0, <3.0.0 Core Kubeflow Pipelines compiler and client runtime
click >=8.0.0 Command-line interface parsing and execution
requests >=2.28.0 HTTP communications with the Aidin Token Exchanger

Next Steps

Section titled “Next Steps”