System Transformers
Introduction
Neosync comes out of the box with 40+ system transformers in order to give you an easy way to get started with anonymizing or generating data. There are two kinds of system transformers:
- Generators - these begin with
generate_
and do not anonymize existing data. They only generate net-new synthetic data. - Transformers - these begin with
transform_
and can anonymize existing data or generate new-new synthetic data based on how you configure it.
Reference
Name | Type | Code | Description |
---|---|---|---|
Generate Categorical | string | Code | Randomly selects a value from a defined set of categorical values |
Generate Email | string | Code | Generates a new randomized email address. |
Transform Email | string | Code | Transforms an existing email address. |
Generate Boolean | boolean | Code | Generates a boolean value at random. |
Generate Card Number | int64 | Code | Generates a card number. |
Generate City | string | Code | Randomly selects a city from a list of predefined US cities. |
Generate E164 Phone Number | string | Code | Generates a Generate phone number in e164 format. |
Generate First Name | string | Code | Generates a random first name. |
Generate Float64 | float64 | Code | Generates a random float64 value. |
Generate Full Address | string | Code | Randomly generates a street address. |
Generate Full Name | string | Code | Generates a new full name consisting of a first and last name. |
Generate Gender | string | Code | Randomly generates one of the following genders: female, male, undefined, nonbinary. |
Generate Int64 Phone Number | int64 | Code | Generates a new phone number of type int64 with a default length of 10. |
Generate Random Int64 | int64 | Code | Generates a random integer value with a default length of 4 unless the Integer Length or Preserve Length parameters are defined. |
Generate Last Name | int64 | Code | Generates a random last name. |
Generate SHA256 Hash | string | Code | SHA256 hashes a randomly generated value. |
Generate SSN | string | Code | Generates a completely random social security numbers including the hyphens in the format xxx-xx-xxxx . |
Generate State | string | Code | Randomly selects a US state and returns the two-character state code. |
Generate Street Address | string | Code | Randomly generates a street address. |
Generate String Phone Number | string | Code | Generates a Generate phone number and returns it as a string. |
Generate Random String | string | Code | Creates a randomly ordered alphanumeric string with a default length of 10 unless the String Length parameter are defined. |
Generate Unix Timestamp | int64 | Code | Randomly generates a Unix timestamp. |
Generate Username | string | Code | Randomly generates a username |
Generate UTC Timestamp | time | Code | Randomly generates a UTC timestamp. |
Generate UUID | uuid | Code | Generates a new UUIDv4 id. |
Generate Zipcode | string | Code | Randomly selects a zip code from a list of predefined US zipcodes. |
Transform E164 Phone Number | string | Code | Transforms an existing E164 formatted phone number. |
Transform First Name | string | Code | Transforms an existing first name |
Transform Float64 | float64 | Code | Transforms an existing float value. |
Transform Full Name | string | Code | Transforms an existing full name. |
Transform Int64 Phone Number | int64 | Code | Transforms an existing phone number that is typed as an integer |
Transform Int64 | int64 | Code | Transforms an existing integer value. |
Transform Last Name | string | Code | Transforms an existing last name. |
Transform Phone Number | string | Code | Transforms an existing phone number that is typed as a string. |
Transform String | string | Code | Transforms an existing string value. |
Passthrough | string | Code | Passes the input value through to the destination with no changes. |
Null | string | Code | Inserts a null string instead of the source value. |
Generate Categorical
The generate categorical transformer allows a user to define a list of comma-separated categorical string values that Neosync will randomly sample.
For example, the following values:
test, me, please
Could produce the following ouput value:
me
Or:
please
Configurations
Name | Description | Default | Example Output |
---|---|---|---|
Categories | list of comma-separated categorical string values that Neosync will randomly sample. | value1,value2 | value1 |
Examples
Categories | Example Input | Example Output |
---|---|---|
false | red,blue,green | green |
false | red,blue,green | red |
Generate Email
The generate email transformer generates a new randomized email address in the format:
<username>@<domain>.<top-level-domain>
By default, the transformer randomizes the username, domain and top-level domain while always preserving the email format by retaining the @
and .
characters.
For example, the following input value:
john@acme.com
Would produce the following ouput value:
ytvub873@gmail.com
Configurations
There are no configurations for this transformer.
Examples
Example Output |
---|
f98723uh@gmail.com |
9cd@msn.com |
mDy@gmail.edu |
fweq23f@hotmail.com |
Transform Email
The transform email transformer can anonymize an existing email address or generate a new randomized email address in the format:
<username>@<domain>.<top-level-domain>
By default, the transformer randomizes the username, domain and top-level domain while always preserving the email format by retaining the @
and .
characters.
For example, the following input value:
john@acme.com
Would produce the following ouput value:
ytvub873@gmail.com
Configurations
Depending on your logic, you may want to configure the output email. The transform email transformer has the following configurations:
Name | Description | Default | Example Input | Example Output |
---|---|---|---|---|
PreserveDomain | Carries over the input domain and top-level to the output email address. | false | john@gmail.com | fjf2903@gmail.com |
PreserveLength | Ensures the output email address is the same length as the input email address. | false | john@gmail.com | hw98@gmail.edu |
Examples
There are several ways you can mix-and-match configurations to get different potential email formats. Here are some possible combinations:
PreserveLength | PreserveDomain | Example Input | Example Output |
---|---|---|---|
false | true | joe@gmail.com | f98723uh@gmail.com |
true | false | joe@gmail.com | 9cd@ms0ej.com |
true | true | joe@gmail.com | mDy@bdo3s.com |
false | false | joe@gmail.com | fweq23f@fheuiw78.edu |
Generate Boolean
The genereate boolean transformer randomly generates a boolean value.
Configurations
There are no configurations for the random boolean transformer.
Examples
Here are some examples of what an output random boolean value may look like.
Example Output |
---|
true |
false` |
Generate Card Number
The generate card number transformer generates a new card number that is luhn valid.
By default, the card number transformer generates a random 16 digit card number that is not luhn valid. If you want luhn validation, please set the luhn-check config to true
.
Configurations
Depending on your validations, you may want to configure the output card number. The card number transformer has the following configurations:
Name | Description | Default | Example Output |
---|---|---|---|
Valid Luhn | Generate a card number that can pass luhn validation. | true | 6621238134011099 |
Examples
Here are some examples of an output card number:
Valid Luhn | Example Output |
---|---|
false | 3627893345931223 |
true | 6621238134011099 |
Generate City
The generate city transformers generates a randomly selected US city. You can see the complete list of cities that are available to be randomly selected here.
If you'd rather not get back a real city value, you can use the Generate Random String Transformer to generate a random string value.
Configurations
There are no configurations for the city transformer.
Examples
Here are some examples of what an output City value may look like.
Example Output |
---|
Louisville |
Manchester |