Documentation

Spreadsheet extends CRM_Import_DataSource
in package

Application

implements DataSourceInterface uses DataSourceTrait

Objects that implement the DataSource interface can be used in CiviCRM imports.

Interfaces

DataSourceInterface: Objects that implement the DataSource interface can be used in CiviCRM imports.

Constants

NUM_ROWS_TO_INSERT = 100

Properties

$dataSourceMetadata : array<string|int, mixed>: Generated metadata relating to the the datasource.
$submittableFields : array<string|int, string>: Form fields declared for this datasource.
$userJob : array<string|int, mixed>: User job details.
$userJobID : int: User job id.

Methods

__construct() : mixed: Class constructor.
buildQuickForm() : void: This is function is called by the form object to get the DataSource's form snippet.
checkPermission() : bool: Determine if the current user has access to this data source.
getAggregateFields() : array<string|int, mixed>|null
getColumnHeaders() : array<string|int, mixed>: Get an array of column headers, if any.
getDataFieldNames() : array<string|int, mixed>: Get the field names of the fields holding data in the import tracking table.
getDataSourceMetadata() : array<string|int, mixed>: Get metadata about the datasource.
getDefaultValues() : array<string|int, mixed>: Get default values for excel dataSource fields.
getInfo() : array<string|int, mixed>: Provides information about the data source.
getLimit() : int
getNumberOfColumns() : int: Get an array of column headers, if any.
getOffset() : int
getRow() : array<string|int, mixed>|null: Get the next row.
getRowCount() : int: Get row count.
getRows() : array<string|int, mixed>: Get rows as an array.
getSelectFields() : array<string|int, mixed>|null
getStandardIndices() : array<string|int, mixed>
getStandardTrackingFields() : array<string|int, mixed>
getStatuses() : array<string|int, mixed>
getSubmittableFields() : array<string|int, mixed>: Get array array of field names that may be submitted for this data source.
getUserJobID() : int|null: Get the ID of the user job being acted on.
initialize() : void: Initialize the datasource, based on the submitted values stored in the user job.
isCompleted() : bool: Has the import job completed.
isValidSpreadsheet() : bool: Is the value in the uploaded file field a valid spreadsheet.
purge() : array<string|int, mixed>: Purge any datasource related assets when the datasource is dropped.
setAggregateFields() : CRM_Import_DataSource
setLimit() : CRM_Import_DataSource
setOffset() : CRM_Import_DataSource
setSelectFields() : CRM_Import_DataSource
setStatuses() : self: Set the statuses to be retrieved.
setUserJobID() : void: Set user job ID.
trimWhitespace() : string
updateStatus() : void: Update the status of the import row to reflect the processing outcome.
addTrackingFieldsToTable() : void: Add a status columns to the import table.
createTempTableFromColumns() : string
getColumnNamesForUnnamedColumns() : array<string|int, mixed>: Get suitable column names for when no header row is in use.
getColumnNamesFromHeaders() : array<string|int, mixed>: Get column names from the headers - munging to lower case etc.
getStatusMapping() : array<string|int, array<string|int, mixed>>: Get the mapping of constants to database status codes.
getSubmittedValue() : mixed: Get submitted value.
getTableName() : string|null: Get the table name for the import job.
getUserJob() : array<string|int, mixed>: Get User Job.
updateUserJobDataSource() : void: Update the data stored in the User Job about the Data Source.
updateUserJobMetadata() : void: Update the UserJob Metadata.
uploadToTable() : array<string|int, mixed>

NUM_ROWS_TO_INSERT


    protected
        mixed
    NUM_ROWS_TO_INSERT
    = 100

$dataSourceMetadata

Generated metadata relating to the the datasource.


        protected
            array<string|int, mixed>
    $dataSourceMetadata
     = []

This is values that are computed within the DataSource class and which are stored in the userJob metadata in the DataSource key - eg.

['table_name' => $]

Will be in the user_job.metadata field encoded into the json like

{'DataSource' : ['table_name' => $], 'submitted_values' : .....}

$submittableFields

Form fields declared for this datasource.


        protected
            array<string|int, string>
    $submittableFields
     = []

$userJob

User job details.


        protected
            array<string|int, mixed>
    $userJob

This is the relevant row from civicrm_user_job.

$userJobID

User job id.


        protected
            int
    $userJobID

This is the primary key of the civicrm_user_job table which is used to track the import.

__construct()

Class constructor.


    public
                    __construct([int|null $userJobID = null ]) : mixed

Parameters

$userJobID : int|null = null

buildQuickForm()

This is function is called by the form object to get the DataSource's form snippet.


    public
                    buildQuickForm(CRM_Contact_Import_Form_DataSource|CRM_Import_Form_DataSourceConfig $form) : void

It should add all fields necessary to get the data uploaded to the temporary table in the DB.

Parameters

$form : CRM_Contact_Import_Form_DataSource|CRM_Import_Form_DataSourceConfig

checkPermission()

Determine if the current user has access to this data source.


    public
                    checkPermission() : bool

Return values

bool

getAggregateFields()


    public
                    getAggregateFields() : array<string|int, mixed>|null

Return values

array<string|int, mixed>|null

getColumnHeaders()

Get an array of column headers, if any.


    public
                    getColumnHeaders() : array<string|int, mixed>

This is presented to the user in the MapField screen so that can see what fields they are mapping.

Return values

array<string|int, mixed>

getDataFieldNames()

Get the field names of the fields holding data in the import tracking table.


    public
                    getDataFieldNames() : array<string|int, mixed>

Return values

array<string|int, mixed>

getDataSourceMetadata()

Get metadata about the datasource.


    public
                    getDataSourceMetadata() : array<string|int, mixed>

Return values

array<string|int, mixed>

getDefaultValues()

Get default values for excel dataSource fields.


    public
                    getDefaultValues() : array<string|int, mixed>

Return values

array<string|int, mixed>

getInfo()

Provides information about the data source.


    public
                    getInfo() : array<string|int, mixed>

Return values

array<string|int, mixed> —

collection of info about this data source

getLimit()


    public
                    getLimit() : int

Return values

int

getNumberOfColumns()

Get an array of column headers, if any.


    public
                    getNumberOfColumns() : int

Null is returned when there are none - ie because a csv file does not have an initial header row.

This is presented to the user in the MapField screen so that can see what fields they are mapping.

Return values

int

getOffset()


    public
                    getOffset() : int

Return values

int

getRow()

Get the next row.


    public
                    getRow() : array<string|int, mixed>|null

Return values

array<string|int, mixed>|null

getRowCount()

Get row count.


    public
                    getRowCount([array<string|int, mixed> $statuses = [] ]) : int

The array has all values.

Parameters

$statuses : array<string|int, mixed> = []

Return values

int

getRows()

Get rows as an array.


    public
                    getRows([bool $nonAssociative = true ]) : array<string|int, mixed>

The array has all values.

Parameters

$nonAssociative : bool = true: Return as a non-associative array?

Return values

array<string|int, mixed>

getSelectFields()


    public
                    getSelectFields() : array<string|int, mixed>|null

Return values

array<string|int, mixed>|null

getStandardIndices()


    public
            static        getStandardIndices() : array<string|int, mixed>

Return values

array<string|int, mixed>

getStandardTrackingFields()


    public
            static        getStandardTrackingFields() : array<string|int, mixed>

Return values

array<string|int, mixed>

getStatuses()


    public
                    getStatuses() : array<string|int, mixed>

Return values

array<string|int, mixed>

getSubmittableFields()

Get array array of field names that may be submitted for this data source.


    public
                    getSubmittableFields() : array<string|int, mixed>

The quick form for the datasource is added by ajax - meaning that QuickForm does not see them as part of the form. However, any fields listed in this array will be taken from the $_POST and stored to the UserJob under the DataSource key.

Return values

array<string|int, mixed>

getUserJobID()

Get the ID of the user job being acted on.


    public
                    getUserJobID() : int|null

Return values

int|null

initialize()

Initialize the datasource, based on the submitted values stored in the user job.


    public
                    initialize() : void

Generally this will include transferring the data to a database table.

isCompleted()

Has the import job completed.


    public
                    isCompleted() : bool

Return values

bool —

True if no rows remain to be imported.

isValidSpreadsheet()

Is the value in the uploaded file field a valid spreadsheet.


    public
            static        isValidSpreadsheet(array<string|int, mixed> $file) : bool

Parameters

$file : array<string|int, mixed>

Return values

bool

purge()

Purge any datasource related assets when the datasource is dropped.


    public
                    purge([array<string|int, mixed> $newParams = [] ]) : array<string|int, mixed>

This is the datasource's chance to delete any tables etc that it created which will now not be used.

Parameters

$newParams : array<string|int, mixed> = []: If the dataSource is being updated to another variant of the same class (eg. the csv upload was set to no column headers and they have resubmitted WITH skipColumnHeader (first row is a header) then the dataSource is still CSV and the params for the new instance are passed in. When changing from csv to SQL (for example) newParams is empty.

Return values

array<string|int, mixed> —

The details to update the DataSource key in the userJob metadata to. Generally and empty array but it the datasource decided (for example) that the table it created earlier is still consistent with the new params then it might decided not to drop the table and would want to retain some metadata.

setAggregateFields()


    public
                    setAggregateFields(array<string|int, mixed> $fields) : CRM_Import_DataSource

Parameters

$fields : array<string|int, mixed>

Return values

CRM_Import_DataSource

setLimit()


    public
                    setLimit(int $limit) : CRM_Import_DataSource

Parameters

$limit : int

Return values

CRM_Import_DataSource

setOffset()


    public
                    setOffset(int $offset) : CRM_Import_DataSource

Parameters

$offset : int

Return values

CRM_Import_DataSource

setSelectFields()


    public
                    setSelectFields(array<string|int, mixed> $selectFields) : CRM_Import_DataSource

Parameters

$selectFields : array<string|int, mixed>

Return values

CRM_Import_DataSource

setStatuses()

Set the statuses to be retrieved.


    public
                    setStatuses(array<string|int, mixed> $statuses) : self

Parameters

$statuses : array<string|int, mixed>

Return values

self

setUserJobID()

Set user job ID.


    public
                    setUserJobID(int $userJobID) : void

Parameters

$userJobID : int

trimWhitespace()


    public
            static        trimWhitespace(string $string) : string

Parameters

$string : string

Return values

string

updateStatus()

Update the status of the import row to reflect the processing outcome.


    public
                    updateStatus(int $id, string $status, string $message[, int|null $entityID = null ][, array<string|int, mixed> $additionalFields = [] ]) : void

Parameters

$id : int
$status : string
$message : string
$entityID : int|null = null: Optional created entity ID
$additionalFields : array<string|int, mixed> = []: Optional array e.g ['related_contact' => 4]

addTrackingFieldsToTable()

Add a status columns to the import table.


    protected
                    addTrackingFieldsToTable(string $tableName) : void

We add _id - primary key _status _statusMsg

Note that

the use of the preceding underscore has 2 purposes - it avoids clashing with an id field (code comments from 14 years ago suggest perhaps there could be cases where it still clashes but time didn't tell in this case)
the show fields query used to get the column names excluded the administrative fields, relying on this convention.

Parameters

$tableName : string

createTempTableFromColumns()


    protected
                    createTempTableFromColumns(array<string|int, mixed> $columns) : string

Parameters

$columns : array<string|int, mixed>

Return values

string —

Temp table name.

getColumnNamesForUnnamedColumns()

Get suitable column names for when no header row is in use.


    protected
                    getColumnNamesForUnnamedColumns(array<string|int, mixed> $row) : array<string|int, mixed>

The result is an array like 'column_1', column_2'. SQL columns cannot start with a number.

Parameters

$row : array<string|int, mixed>

Return values

array<string|int, mixed>

getColumnNamesFromHeaders()

Get column names from the headers - munging to lower case etc.


    protected
                    getColumnNamesFromHeaders(array<string|int, mixed> $headers) : array<string|int, mixed>

Parameters

$headers : array<string|int, mixed>

Return values

array<string|int, mixed>

getStatusMapping()

Get the mapping of constants to database status codes.


    protected
                    getStatusMapping() : array<string|int, array<string|int, mixed>>

Return values

array<string|int, array<string|int, mixed>>

getSubmittedValue()

Get submitted value.


    protected
                    getSubmittedValue(string $valueName) : mixed

Get a value submitted on the form.

Parameters

$valueName : string

getTableName()

Get the table name for the import job.


    protected
                    getTableName() : string|null

Return values

string|null

getUserJob()

Get User Job.


    protected
                    getUserJob() : array<string|int, mixed>

API call to retrieve the userJob row.

Return values

array<string|int, mixed>

updateUserJobDataSource()

Update the data stored in the User Job about the Data Source.


    protected
                    updateUserJobDataSource(array<string|int, mixed> $data) : void

Parameters

$data : array<string|int, mixed>

updateUserJobMetadata()

Update the UserJob Metadata.


    private
                    updateUserJobMetadata(string $key, array<string|int, mixed> $data) : void

Parameters

$key : string
$data : array<string|int, mixed>

uploadToTable()


    private
                    uploadToTable() : array<string|int, mixed>

Return values

array<string|int, mixed>

Spreadsheet extends CRM_Import_DataSource in package Application implements DataSourceInterface uses DataSourceTrait

Table of Contents

Interfaces

Constants

Properties

Methods

Constants

NUM_ROWS_TO_INSERT

Properties

$dataSourceMetadata

$submittableFields

$userJob

$userJobID

Methods

__construct()

Parameters

buildQuickForm()

Parameters

Tags

checkPermission()

Return values

getAggregateFields()

Return values

getColumnHeaders()

Tags

Return values

getDataFieldNames()

Tags

Return values

getDataSourceMetadata()

Tags

Return values

getDefaultValues()

Return values

getInfo()

Return values

getLimit()

Return values

getNumberOfColumns()

Tags

Return values

getOffset()

Return values

getRow()

Tags

Return values

getRowCount()

Parameters

Tags

Return values

getRows()

Parameters

Tags

Return values

getSelectFields()

Return values

getStandardIndices()

Return values

getStandardTrackingFields()

Return values

getStatuses()

Return values

getSubmittableFields()

Return values

getUserJobID()

Return values

initialize()

Tags

isCompleted()

Tags

Return values

isValidSpreadsheet()

Parameters

Tags

Return values

purge()

Parameters

Tags

Return values

setAggregateFields()

Spreadsheet extends CRM_Import_DataSource
in package

Application

implements DataSourceInterface uses DataSourceTrait