Custom data migration services

Suggest Edits

Most Identity Cloud deployments include importing user records from the customer’s data store into the Akamai platform. The Custom Data Migration service allows for migrating user records into the Akamai user profile data store from one or more CSV (comma-separated value) files.

This Custom Data Migration service may include one or more of the following services as defined and scoped within a Statement of Work:

Importing more than 10 million user records
Importing records with a throughput of more than 1 million user records created per hour
Data mapping from legacy system fields to Identity Cloud attributes
Data transformation from the legacy system types and formats into Identity Cloud types and formats
De-duplication of user profile records
Importing one-to-many relational data into Identity Cloud plural attributes
Importing social profile identifiers
Delta migrations to minimize production downtime and/or onboard web properties in a phased approach

This service consists of two or more phases. A sample migration is performed to develop the tools and processes and verify that data is correctly exported, transformed, and imported. A production migration is scheduled immediately prior to a production launch to import the full set of data. One delta migration may be scheduled to reconcile records changed during the production migration window or add additional data sets in a phased approach. Delta migrations must be scoped within a Statement of Work.

The sample migration process

Customer exports a representative sample of data to one or more CSV files.
Customer encrypts the CSV file and provides the passphrase to an Akamai Technical Lead via a secure channel.
Customer uploads the CSV file to Akamai's secure FTP server.
Customer works with an Akamai Technical Lead to define:
- Data mapping (if applicable). Data mapping indicates which fields in the customer database correspond to the attributes used in an Identity Cloud user profile. For example, the data map might specify that the LastName field is equivalent to the familyName user profile attribute.
- Data transformations (if applicable). Data transformations convert data from one format to another. For example, suppose you have data stored in a Yes/No field that must be converted to a Boolean type (True/False). A data transformation would convert each Yes value to True and each No value to False.
- De-duplication business rules (if applicable)
- Delta update logic (if applicable)
Akamai develops custom software to apply mappings, transformations, de-duplication logic, and delta updates as scoped within a Statement of Work.
Akamai imports the sample data into the development environment.
Customer validates the sample data imported by Akamai.

📘
Once the sample migration has been completed and validated, the format of the data CSV files cannot change.

The production migration process

Customer exports data to a CSV file using the same process/tools used during sample migration.
Customer encrypts the CSV file using the same passphrase used during sample migration.
Customer uploads the CSV file to Akamai's secure FTP server.
Akamai imports the production data to the production environment.
Customer validates production data imported by Akamai.

The delta migration process

Customer exports data to a CSV file using the same process/tools used during sample migration.
Customer encrypts the CSV file using the same passphrase used during sample migration.
Customer uploads the CSV file to Akamai's secure FTP server.
Akamai performs import of production data to production environment:
- New records are created
- Existing records are updated according to defined business rules
Customer validates the production data imported by Akamai.

Data format

The user profile data must be provided in valid CSV-formatted files as defined by RFC 4180:

Fields must be delimited by commas.
Fields containing commas or line breaks must be enclosed in double quotes.
Double quotes within the field must be escaped with an additional double quote. For example:
```
"This is a field with ""quoted"" text."
```
The file must be UTF-8 encoded.
The first row of data must be a header row, which contains the names of the schema attributes to which the corresponding fields will be mapped.
Attributes within objects must be specified with a period delimiter, also known as “dot notation”. For example:
```
primaryAddress.city
```
Boolean data must be represented as true and false. (Although this is case insensitive: you can also specify Boolean data as True and False or TRUE and FALSE.)
Gender data must be represented as male and female. (Although this is case insensitive: you can also specify gender data as Male and Female or MALE and FEMALE.
Date and time data must be represented as a UTC timestamp in the format: yy-mm-dd hh:mm:ss. For example:
```
2014-01-15 14:30:00
```

Data de-duplication

De-duplication of user records in a source file can be complex. Each customer must work with an Akamai Technical Lead to define the business logic to handle collisions in data. Considerations include:

What data uniquely identifies the record?
Is there a timestamp that can be used to determine which data is more recent?
When merging duplicate records, should blank values in the CSV be overwritten, ignored, or set to NULL?
Should plural elements be merged, updated, or replaced?

Encrypting data

Files containing sensitive data must be encrypted prior to uploading to Akamai's secure FTP server. Data can be encrypted using 256-bit Advanced Encryption Standard and the GNU Privacy Guard application. To encrypt your data, run GNU Privacy Guard from the command line, using a command similar to the following:

gpg --symmetric --cipher-algo aes256 data.csv

Choose a passphrase with high entropy such as five random words and communicate the passphrase to the Akamai Technical Lead via phone. Do not send the passphrase in an email nor in a support ticket.

Secure File Transfer (sFTP)

Akamai will configure a temporary sFTP account for uploading encrypted data files. The customer must provide the Akamai Technical Lead with the following information:

A list of all IP addresses that will connect to the sFTP server
Public SSH keys for all users that will connect to the sFTP server

📘
Password authentication to the sFTP server can be provided for customers who are not able to use SSH.

Reports

Akamai will deliver one file after importing data. All records are logged in a CSV file containing:

The line number from the source data file
The UUID of the record in the Identity Cloud user profile database if the record was imported successfully
The error message if the record was not imported successfully

Updated almost 3 years ago