Documentation

CRM_Import_DataSource_CSV extends CRM_Import_DataSource
in package

This class defines the DataSource interface but must be subclassed to be useful.

Tags
copyright

CiviCRM LLC https://civicrm.org/licensing

Table of Contents

Constants

NUM_ROWS_TO_INSERT  = 100

Properties

$dataSourceMetadata  : array<string|int, mixed>
Generated metadata relating to the the datasource.
$submittableFields  : array<string|int, string>
Form fields declared for this datasource.
$userJob  : array<string|int, mixed>
User job details.
$userJobID  : int
User job id.
$aggregateFields  : array<string|int, mixed>
Fields to select as aggregates.
$limit  : int
$offset  : int
$queryResultObject  : CRM_Core_DAO
$row  : array<string|int, mixed>
Current row.
$selectFields  : array<string|int, mixed>
Fields to select.
$statuses  : array<string|int, mixed>
Statuses of rows to fetch.
$tableName  : string
The name of the import table.

Methods

__construct()  : mixed
Class constructor.
buildQuickForm()  : void
This is function is called by the form object to get the DataSource's form snippet.
checkPermission()  : bool
Determine if the current user has access to this data source.
getAggregateFields()  : array<string|int, mixed>|null
getColumnHeaders()  : array<string|int, mixed>
Get an array of column headers, if any.
getDataFieldNames()  : array<string|int, mixed>
Get the field names of the fields holding data in the import tracking table.
getDataSourceMetadata()  : array<string|int, mixed>
Get metadata about the datasource.
getDefaultValues()  : array<string|int, mixed>
Get default values for csv dataSource fields.
getInfo()  : array<string|int, mixed>
Provides information about the data source.
getLimit()  : int
getNumberOfColumns()  : int
Get an array of column headers, if any.
getOffset()  : int
getRow()  : array<string|int, mixed>|null
Get the next row.
getRowCount()  : int
Get row count.
getRows()  : array<string|int, mixed>
Get rows as an array.
getSelectFields()  : array<string|int, mixed>|null
getStatuses()  : array<string|int, mixed>
getSubmittableFields()  : array<string|int, string>
Get the fields declared for this datasource.
getUserJobID()  : int|null
Get the ID of the user job being acted on.
initialize()  : void
Initialize the datasource, based on the submitted values stored in the user job.
isCompleted()  : bool
Has the import job completed.
purge()  : array<string|int, mixed>
Purge any datasource related assets when the datasource is dropped.
setAggregateFields()  : CRM_Import_DataSource
setLimit()  : CRM_Import_DataSource
setOffset()  : CRM_Import_DataSource
setSelectFields()  : CRM_Import_DataSource
setStatuses()  : self
Set the statuses to be retrieved.
setUserJobID()  : void
Set user job ID.
trimNonBreakingSpaces()  : string
Trim non-breaking spaces in a multibyte-safe way.
updateStatus()  : void
Update the status of the import row to reflect the processing outcome.
addTrackingFieldsToTable()  : void
Add a status columns to the import table.
createTempTableFromColumns()  : string
getColumnNamesForUnnamedColumns()  : array<string|int, mixed>
Get suitable column names for when no header row is in use.
getColumnNamesFromHeaders()  : array<string|int, mixed>
Get column names from the headers - munging to lower case etc.
getStatusMapping()  : array<string|int, array<string|int, mixed>>
Get the mapping of constants to database status codes.
getSubmittedValue()  : mixed
Get submitted value.
getTableName()  : string|null
Get the table name for the import job.
getUserJob()  : array<string|int, mixed>
Get User Job.
updateUserJobDataSource()  : void
Update the data stored in the User Job about the Data Source.
csvToTable()  : array<string|int, mixed>
Create a table that matches the CSV file and populate it with the file's contents
flushQueryResults()  : void
Flush the existing query to retrieve rows.
getAdditionalTrackingFields()  : string
Get any additional import specific tracking fields.
getParser()  : CRM_Import_Parser
Get the import parser.
getSelectClause()  : string
getStatusClause()  : string
Get the status filter clause.
instantiateQueryObject()  : void
updateUserJobMetadata()  : void
Update the UserJob Metadata.

Constants

NUM_ROWS_TO_INSERT

private mixed NUM_ROWS_TO_INSERT = 100

Properties

$dataSourceMetadata

Generated metadata relating to the the datasource.

protected array<string|int, mixed> $dataSourceMetadata = []

This is values that are computed within the DataSource class and which are stored in the userJob metadata in the DataSource key - eg.

['table_name' => $]

Will be in the user_job.metadata field encoded into the json like

{'DataSource' : ['table_name' => $], 'submitted_values' : .....}

$submittableFields

Form fields declared for this datasource.

protected array<string|int, string> $submittableFields = ['skipColumnHeader', 'uploadFile', 'fieldSeparator']

$userJob

User job details.

protected array<string|int, mixed> $userJob

This is the relevant row from civicrm_user_job.

$userJobID

User job id.

protected int $userJobID

This is the primary key of the civicrm_user_job table which is used to track the import.

$aggregateFields

Fields to select as aggregates.

private array<string|int, mixed> $aggregateFields

$row

Current row.

private array<string|int, mixed> $row

$selectFields

Fields to select.

private array<string|int, mixed> $selectFields

$statuses

Statuses of rows to fetch.

private array<string|int, mixed> $statuses = []

$tableName

The name of the import table.

private string $tableName

Methods

__construct()

Class constructor.

public __construct([int|null $userJobID = NULL ]) : mixed
Parameters
$userJobID : int|null = NULL

checkPermission()

Determine if the current user has access to this data source.

public checkPermission() : bool
Return values
bool

getAggregateFields()

public getAggregateFields() : array<string|int, mixed>|null
Return values
array<string|int, mixed>|null

getColumnHeaders()

Get an array of column headers, if any.

public getColumnHeaders() : array<string|int, mixed>

This is presented to the user in the MapField screen so that can see what fields they are mapping.

Tags
throws
CRM_Core_Exception
Return values
array<string|int, mixed>

getDataFieldNames()

Get the field names of the fields holding data in the import tracking table.

public getDataFieldNames() : array<string|int, mixed>
Tags
throws
CRM_Core_Exception
Return values
array<string|int, mixed>

getDataSourceMetadata()

Get metadata about the datasource.

public getDataSourceMetadata() : array<string|int, mixed>
Tags
throws
CRM_Core_Exception
Return values
array<string|int, mixed>

getDefaultValues()

Get default values for csv dataSource fields.

public getDefaultValues() : array<string|int, mixed>
Return values
array<string|int, mixed>

getInfo()

Provides information about the data source.

public getInfo() : array<string|int, mixed>
Return values
array<string|int, mixed> —

collection of info about this data source

getLimit()

public getLimit() : int
Return values
int

getNumberOfColumns()

Get an array of column headers, if any.

public getNumberOfColumns() : int

Null is returned when there are none - ie because a csv file does not have an initial header row.

This is presented to the user in the MapField screen so that can see what fields they are mapping.

Tags
throws
CRM_Core_Exception
Return values
int

getOffset()

public getOffset() : int
Return values
int

getRow()

Get the next row.

public getRow() : array<string|int, mixed>|null
Tags
throws
CRM_Core_Exception
Return values
array<string|int, mixed>|null

getRowCount()

Get row count.

public getRowCount([array<string|int, mixed> $statuses = [] ]) : int

The array has all values.

Parameters
$statuses : array<string|int, mixed> = []
Tags
throws
CRM_Core_Exception
Return values
int

getRows()

Get rows as an array.

public getRows([bool $nonAssociative = TRUE ]) : array<string|int, mixed>

The array has all values.

Parameters
$nonAssociative : bool = TRUE

Return as a non-associative array?

Tags
throws
CRM_Core_Exception
Return values
array<string|int, mixed>

getSelectFields()

public getSelectFields() : array<string|int, mixed>|null
Return values
array<string|int, mixed>|null

getStatuses()

public getStatuses() : array<string|int, mixed>
Return values
array<string|int, mixed>

getSubmittableFields()

Get the fields declared for this datasource.

public getSubmittableFields() : array<string|int, string>
Return values
array<string|int, string>

getUserJobID()

Get the ID of the user job being acted on.

public getUserJobID() : int|null
Return values
int|null

initialize()

Initialize the datasource, based on the submitted values stored in the user job.

public initialize() : void
Tags
throws
CRM_Core_Exception

isCompleted()

Has the import job completed.

public isCompleted() : bool
Tags
throws
CRM_Core_Exception
Return values
bool —

True if no rows remain to be imported.

purge()

Purge any datasource related assets when the datasource is dropped.

public purge([array<string|int, mixed> $newParams = [] ]) : array<string|int, mixed>

This is the datasource's chance to delete any tables etc that it created which will now not be used.

Parameters
$newParams : array<string|int, mixed> = []

If the dataSource is being updated to another variant of the same class (eg. the csv upload was set to no column headers and they have resubmitted WITH skipColumnHeader (first row is a header) then the dataSource is still CSV and the params for the new instance are passed in. When changing from csv to SQL (for example) newParams is empty.

Tags
throws
CRM_Core_Exception
Return values
array<string|int, mixed> —

The details to update the DataSource key in the userJob metadata to. Generally and empty array but it the datasource decided (for example) that the table it created earlier is still consistent with the new params then it might decided not to drop the table and would want to retain some metadata.

setStatuses()

Set the statuses to be retrieved.

public setStatuses(array<string|int, mixed> $statuses) : self
Parameters
$statuses : array<string|int, mixed>
Return values
self

setUserJobID()

Set user job ID.

public setUserJobID(int $userJobID) : void
Parameters
$userJobID : int

trimNonBreakingSpaces()

Trim non-breaking spaces in a multibyte-safe way.

public static trimNonBreakingSpaces(string $string) : string

See also dev/core#2127 - avoid breaking strings ending in à or any other unicode character sharing the same 0xA0 byte as a non-breaking space.

Parameters
$string : string
Return values
string —

The trimmed string

updateStatus()

Update the status of the import row to reflect the processing outcome.

public updateStatus(int $id, string $status, string $message[, int|null $entityID = NULL ][, array<string|int, mixed> $additionalFields = [] ]) : void
Parameters
$id : int
$status : string
$message : string
$entityID : int|null = NULL

Optional created entity ID

$additionalFields : array<string|int, mixed> = []

Optional array e.g ['related_contact' => 4]

Tags
throws
CRM_Core_Exception

addTrackingFieldsToTable()

Add a status columns to the import table.

protected addTrackingFieldsToTable(string $tableName) : void

We add _id - primary key _status _statusMsg

Note that

  1. the use of the preceding underscore has 2 purposes - it avoids clashing with an id field (code comments from 14 years ago suggest perhaps there could be cases where it still clashes but time didn't tell in this case)
  2. the show fields query used to get the column names excluded the administrative fields, relying on this convention.
  3. we have the capitalisation on _statusMsg - @param string $tableName
Parameters
$tableName : string
Tags
throws
CRM_Core_Exception
todo

change to _status_message

createTempTableFromColumns()

protected createTempTableFromColumns(array<string|int, mixed> $columns) : string
Parameters
$columns : array<string|int, mixed>
Tags
throws
DBQueryException
Return values
string —

Temp table name.

getColumnNamesForUnnamedColumns()

Get suitable column names for when no header row is in use.

protected getColumnNamesForUnnamedColumns(array<string|int, mixed> $row) : array<string|int, mixed>

The result is an array like 'column_1', column_2'. SQL columns cannot start with a number.

Parameters
$row : array<string|int, mixed>
Return values
array<string|int, mixed>

getColumnNamesFromHeaders()

Get column names from the headers - munging to lower case etc.

protected getColumnNamesFromHeaders(array<string|int, mixed> $headers) : array<string|int, mixed>
Parameters
$headers : array<string|int, mixed>
Return values
array<string|int, mixed>

getStatusMapping()

Get the mapping of constants to database status codes.

protected getStatusMapping() : array<string|int, array<string|int, mixed>>
Return values
array<string|int, array<string|int, mixed>>

getSubmittedValue()

Get submitted value.

protected getSubmittedValue(string $valueName) : mixed

Get a value submitted on the form.

Parameters
$valueName : string
Tags
throws
CRM_Core_Exception

getTableName()

Get the table name for the import job.

protected getTableName() : string|null
Tags
throws
CRM_Core_Exception
Return values
string|null

getUserJob()

Get User Job.

protected getUserJob() : array<string|int, mixed>

API call to retrieve the userJob row.

Tags
throws
CRM_Core_Exception
Return values
array<string|int, mixed>

updateUserJobDataSource()

Update the data stored in the User Job about the Data Source.

protected updateUserJobDataSource(array<string|int, mixed> $data) : void
Parameters
$data : array<string|int, mixed>
Tags
throws
CRM_Core_Exception
throws
UnauthorizedException

csvToTable()

Create a table that matches the CSV file and populate it with the file's contents

private csvToTable(string $file[, bool $headers = FALSE ][, string $fieldSeparator = ',' ]) : array<string|int, mixed>
Parameters
$file : string

File name to load.

$headers : bool = FALSE

Whether the first row contains headers.

$fieldSeparator : string = ','

Character that separates the various columns in the file.

Tags
throws
CRM_Core_Exception
Return values
array<string|int, mixed> —

name of the created table

flushQueryResults()

Flush the existing query to retrieve rows.

private flushQueryResults() : void

The query will be run again, potentially retrieving newly-available rows. Note the 'newly available' could mean an external process has intervened. For example the import_extensions lazy-loads into the import table.

getAdditionalTrackingFields()

Get any additional import specific tracking fields.

private getAdditionalTrackingFields() : string
Tags
throws
CRM_Core_Exception
Return values
string

getParser()

Get the import parser.

private getParser() : CRM_Import_Parser
Tags
throws
CRM_Core_Exception
Return values
CRM_Import_Parser

getSelectClause()

private getSelectClause() : string
Return values
string

getStatusClause()

Get the status filter clause.

private getStatusClause() : string
Return values
string 

        

        
    




    

    

                                

                

                                
    

        On this page

        
    


                

                            

            

    

        

            
Search results