Transform Stage
The Transform stage is the final step in the Pipe lifecycle. It is reached only if all Conditions and Matches have passed.
It modifies the value into its final form before it is returned by the pipeline or passed to your function.
Data Integrity
Because Transformation happens last, you are guaranteed to be transforming data that has already been verified as valid, preventing errors like trying to .strip() an Integer or a None value.
Setup vs Transform: When to Use Each
Both setup and transform use transform handlers, but they run at different stages of the Pipe lifecycle:
| Feature | Setup | Transform |
|---|---|---|
| When it runs | After type validation, before conditions/matches | After all validations pass (final stage) |
| Purpose | Data preparation and normalization | Data modification and formatting |
| Validation | Only type is validated | All conditions and matches have passed |
| Use case | Strip whitespace, normalize input | Format output, apply business logic |
Example: Setup for Preparation
Use setup when you need to normalize data before validation:
from pipeline import Pipe
# Setup strips whitespace BEFORE checking email format
result = Pipe(
value=" user@example.com ",
type=str,
setup={Pipe.Transform.Strip: None}, # Runs first
matches={Pipe.Match.Format.Email: None}, # Validates cleaned value
).run()
print(result.value) # "user@example.com" (no spaces)
Example: Transform for Formatting
Use transform when you need to modify data after validation:
from pipeline import Pipe
# Transform lowercases AFTER validation passes
result = Pipe(
value="User@Example.COM",
type=str,
matches={Pipe.Match.Format.Email: None}, # Validates first
transform={Pipe.Transform.Lowercase: None} # Formats after validation
).run()
print(result.value) # "user@example.com"
Combining Setup and Transform
You can use both together for a complete data processing pipeline:
from pipeline import Pipe
result = Pipe(
value=" User@Example.COM ",
type=str,
setup={Pipe.Transform.Strip: None}, # 1. Clean input
conditions={Pipe.Condition.MaxLength: 50}, # 2. Validate structure
matches={Pipe.Match.Format.Email: None}, # 3. Validate format
transform={Pipe.Transform.Lowercase: None} # 4. Format output
).run()
print(result.value) # "user@example.com"
Setup vs Pre-Hook
While pre_hook can perform the same data preparation as setup, the setup argument is a developer-friendly simplification. Important: setup is per-pipe (field-specific), while pre_hook is global (applies to all fields in a Pipeline). Use setup for field-specific transformations and pre_hook for global logic or complex custom logic.
Advanced: Custom Transformation Logic
While transformations are powerful, sometimes you need custom logic. Use hooks for this:
from pipeline import Pipeline, Pipe
# Create pipeline
user_pipeline = Pipeline(
name={"type": str},
email={"type": str}
)
# Add custom transformation via pre_hook
def custom_transform(hook):
current_value = hook.value.get
if hook.field == "name":
# Custom logic: Title case
hook.value.set(current_value.title())
elif hook.field == "email":
# Custom logic: Lowercase and strip
hook.value.set(current_value.lower().strip())
user_pipeline.pre_hook = custom_transform
result = user_pipeline.run(data={
"name": "john doe",
"email": " User@Example.COM "
})
print(result.processed_data)
# {
# 'name': 'John Doe',
# 'email': 'user@example.com'
# }
Best Practices
Order Matters
Transformations are applied in the order they're defined:
transform={
Pipe.Transform.Strip: None, # 1. Remove whitespace
Pipe.Transform.Lowercase: None, # 2. Convert to lowercase
Pipe.Transform.Capitalize: None # 3. Capitalize
}
Combine with Validation
Always validate before transforming:
email={
"type": str,
"conditions": {Pipe.Condition.MaxLength: 64}, # Validate
"matches": {Pipe.Match.Format.Email: None}, # Validate
"transform": {Pipe.Transform.Lowercase: None} # Transform
}
Type Safety
Transformations only run if validation passes, ensuring type safety:
# If value is not a string, Lowercase won't run
Pipe(
value=123, # Wrong type
type=str,
transform={Pipe.Transform.Lowercase: None}
)
Technical Reference
The following section is automatically generated from the source code, detailing transformation handlers like string manipulation, math operations, and unique filtering.
Transform
Central registry for all transform handlers.
Transform handlers modify the value in some way, such as changing case, replacing substrings, or performing arithmetic operations.
Source code in pipeline/handlers/transform_handler/transform.py
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 | |
Apply
Bases: TransformHandler[Any, Callable]
Transforms the value by passing it through the provided callable.
The callable should take only one argument (the value) and return the transformed value. No checks are performed on the returned value, so ensure it meets your specific requirements.
Source code in pipeline/handlers/transform_handler/transform.py
97 98 99 100 101 102 103 104 105 106 107 108 | |
Capitalize
Bases: TransformHandler[str, None]
Converts the first character to uppercase and the rest to lowercase
Source code in pipeline/handlers/transform_handler/transform.py
23 24 25 26 27 28 | |
Lowercase
Bases: TransformHandler[str, None]
Converts all characters in the string to lowercase
Source code in pipeline/handlers/transform_handler/transform.py
30 31 32 33 34 35 | |
Multiply
Bases: TransformHandler[str | list | int | float, int | float]
Multiplies a string, list, integer or float by the provided argument
If the value is not numeric, the argument is rounded before multiplication.
Source code in pipeline/handlers/transform_handler/transform.py
44 45 46 47 48 49 50 51 52 53 54 55 56 | |
Replace
Bases: TransformHandler[str, tuple]
Replaces all occurrences of a substring with another (old, new)
Source code in pipeline/handlers/transform_handler/transform.py
88 89 90 91 92 93 94 95 | |
Reverse
Bases: TransformHandler[str | list, None]
Reverses the order of characters in a string or items in a list
Source code in pipeline/handlers/transform_handler/transform.py
58 59 60 61 62 63 | |
SnakeCase
Bases: TransformHandler[str, None]
Converts strings to snake_case (e.g., 'HelloWorld' -> 'hello_world')
Source code in pipeline/handlers/transform_handler/transform.py
72 73 74 75 76 77 78 79 | |
Strip
Bases: TransformHandler[str, Optional[str]]
Removes leading and trailing whitespace from a string
Source code in pipeline/handlers/transform_handler/transform.py
16 17 18 19 20 21 | |
Title
Bases: TransformHandler[str, None]
Converts the first character of every word to uppercase
Source code in pipeline/handlers/transform_handler/transform.py
65 66 67 68 69 70 | |
Unique
Bases: TransformHandler[list, None]
Removes duplicate items from a list while preserving order
Source code in pipeline/handlers/transform_handler/transform.py
81 82 83 84 85 86 | |
Uppercase
Bases: TransformHandler[str, None]
Converts all characters in the string to uppercase
Source code in pipeline/handlers/transform_handler/transform.py
37 38 39 40 41 42 | |