eu.ehri.extension

Class ImportResource

java.lang.Object

eu.ehri.extension.base.AbstractResource

eu.ehri.extension.ImportResource

All Implemented Interfaces:

TxCheckedResource

public class ImportResource
extends AbstractResource

Resource class for import endpoints.

Field Summary

Fields

Modifier and Type	Field and Description
static String	ALLOW_UPDATES_PARAM
static String	BASE_URI_PARAM
static String	ENDPOINT
static String	FORMAT_PARAM
static String	HANDLER_PARAM
static String	IMPORTER_PARAM
static String	PROPERTIES_PARAM
static String	URI_SUFFIX_PARAM

Fields inherited from class eu.ehri.extension.base.AbstractResource

ACCESSOR_PARAM, ALL_PARAM, AUTH_HEADER_NAME, COMMIT_PARAM, CSV_MEDIA_TYPE, DEFAULT_LIST_LIMIT, FILTER_PARAM, graph, GROUP_PARAM, ID_PARAM, INCLUDE_PROPS_PARAM, ITEM_CACHE_TIME, jsonFactory, jsonMapper, LIMIT_PARAM, LOG_MESSAGE_HEADER_NAME, LOG_PARAM, manager, N3_MIMETYPE, OFFSET_PARAM, PATCH_HEADER_NAME, RANGE_HEADER_NAME, RDF_MIMETYPE_FORMATS, RDF_XML_MIMETYPE, request, requestHeaders, RESOURCE_ENDPOINT_PREFIX, SCOPE_PARAM, SORT_PARAM, STREAM_HEADER_NAME, TOLERANT_PARAM, TURTLE_MIMETYPE, uriInfo, VERSION_PARAM

Constructor Summary

Constructors

Constructor and Description
ImportResource(org.neo4j.graphdb.GraphDatabaseService database)

Method Summary

Modifier and Type	Method and Description
ImportLog	importCsv(String scopeId, Boolean tolerant, Boolean allowUpdates, String logMessage, String importerClass, boolean commit, InputStream data) Import a set of CSV files.
ImportLog	importEac(String scopeId, Boolean tolerant, Boolean allowUpdates, String logMessage, String propertyFile, String handlerClass, String importerClass, boolean commit, InputStream data) Import EAC files.
ImportLog	importEad(String scopeId, Boolean tolerant, Boolean allowUpdates, String logMessage, String propertyFile, String handlerClass, String importerClass, boolean commit, InputStream data) Import a set of EAD files.
ImportLog	importEag(String scopeId, Boolean tolerant, Boolean allowUpdates, String logMessage, String propertyFile, String handlerClass, String importerClass, boolean commit, InputStream data) Import EAG files.
ImportLog	importLinks(Boolean tolerant, boolean commit, Table table) Create multiple links via CSV or JSON tabular upload.
ImportLog	importSkos(String scopeId, Boolean tolerant, String baseURI, String uriSuffix, String logMessage, String format, boolean commit, InputStream stream) Import a SKOS file, of varying formats, as specified by the "language" column of the file extensions table here.
SyncLog	syncEad(String scopeId, String fonds, Boolean tolerant, Boolean allowUpdates, String logMessage, String propertyFile, String handlerClass, String importerClass, Set<String> ex, boolean commit, InputStream data) Synchronise a repository or fonds via EAD data, removing items which are not present in the incoming data.

Methods inherited from class eu.ehri.extension.base.AbstractResource

anonymousApi, api, beginTx, creationResponse, getCacheControl, getCurrentActioner, getCurrentUser, getGraph, getIntQueryParam, getItemUri, getLogMessage, getQuery, getRdfFormat, getRequesterUserProfile, getSerializer, getStringListQueryParam, isPatch, isStreaming, single, streamingList, streamingList, streamingListOfLists, streamingPage, streamingPage, streamingVertexList, streamingVertexList

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ENDPOINT

public static final String ENDPOINT

See Also:

Constant Field Values

BASE_URI_PARAM

public static final String BASE_URI_PARAM

See Also:

Constant Field Values

URI_SUFFIX_PARAM

public static final String URI_SUFFIX_PARAM

See Also:

Constant Field Values

ALLOW_UPDATES_PARAM

public static final String ALLOW_UPDATES_PARAM

See Also:

Constant Field Values

HANDLER_PARAM

public static final String HANDLER_PARAM

See Also:

Constant Field Values

IMPORTER_PARAM

public static final String IMPORTER_PARAM

See Also:

Constant Field Values

PROPERTIES_PARAM

public static final String PROPERTIES_PARAM

See Also:

Constant Field Values

FORMAT_PARAM

public static final String FORMAT_PARAM

See Also:

Constant Field Values

Constructor Detail

ImportResource

public ImportResource(@Context
                      org.neo4j.graphdb.GraphDatabaseService database)

Method Detail

importSkos

public ImportLog importSkos(String scopeId,
                            Boolean tolerant,
                            String baseURI,
                            String uriSuffix,
                            String logMessage,
                            String format,
                            boolean commit,
                            InputStream stream)
                     throws ItemNotFound,
                            ValidationError,
                            IOException,
                            DeserializationError

Import a SKOS file, of varying formats, as specified by the "language" column of the file extensions table here.

Example:

    
 curl -X POST \
      -H "X-User: mike" \
      --data-binary @skos-data.rdf \
      "http://localhost:7474/ehri/import/skos?scope=gb-my-vocabulary&log=testing&tolerant=true"
     
 

Parameters:

scopeId - the id of the import scope (i.e. repository)

tolerant - whether or not to die on the first validation error

logMessage - log message for import. If this refers to an accessible local file its contents will be used.

baseURI - a URI prefix common to ingested items that will be removed from each item's URI to obtain the local identifier.

uriSuffix - a URI suffix common to ingested items that will be removed from each item's URI to obtain the local identifier.

format - the RDF format of the POSTed data

commit - commit the operation to the database. The default mode is to operate as a dry-run

stream - a stream of SKOS data in a valid format.

Returns:

a JSON object showing how many records were created, updated, or unchanged.

Throws:

ItemNotFound

ValidationError

IOException

DeserializationError

importEad

public ImportLog importEad(String scopeId,
                           Boolean tolerant,
                           Boolean allowUpdates,
                           String logMessage,
                           String propertyFile,
                           String handlerClass,
                           String importerClass,
                           boolean commit,
                           InputStream data)
                    throws ItemNotFound,
                           ValidationError,
                           IOException,
                           DeserializationError

Import a set of EAD files. The POST body can be one of:

• a single EAD file
• multiple EAD files in an archive such as tar, tar.gz, or zip
• a plain test file containing local file paths

The Content-Type header is used to distinguish the contents.

The way you would run with would typically be:

    
     curl -X POST \
      -H "X-User: mike" \
      --data-binary @ead-list.txt \
      "http://localhost:7474/ehri/import/ead?scope=my-repo-id&log=testing&tolerant=true"

 # NB: Data is sent using --data-binary to preserve line-breaks - otherwise
 # it needs url encoding.
    
 

(Assuming ead-list.txt is a list of newline separated EAD file paths.)

(TODO: Might be better to use a different way of encoding the local file paths...)

Parameters:

scopeId - the id of the import scope (i.e. repository)

tolerant - whether or not to die on the first validation error

allowUpdates - allow the importer to update items that already exist. If it attempts to do so without this option enabled an error will be thrown

logMessage - log message for import. If this refers to an accessible local file its contents will be used.

handlerClass - the fully-qualified handler class name (defaults to EadHandler)

importerClass - the fully-qualified import class name (defaults to EadImporter)

propertyFile - a local file path pointing to an import properties configuration file.

commit - commit the operation to the database. The default mode is to operate as a dry-run

data - file data containing one of: a single EAD file, multiple EAD files in an archive, a list of local file paths. The Content-Type header is used to distinguish the contents.

Returns:

a JSON object showing how many records were created, updated, or unchanged.

Throws:

ItemNotFound

ValidationError

IOException

DeserializationError

syncEad

public SyncLog syncEad(String scopeId,
                       String fonds,
                       Boolean tolerant,
                       Boolean allowUpdates,
                       String logMessage,
                       String propertyFile,
                       String handlerClass,
                       String importerClass,
                       Set<String> ex,
                       boolean commit,
                       InputStream data)
                throws ItemNotFound,
                       ValidationError,
                       IOException,
                       DeserializationError

Synchronise a repository or fonds via EAD data, removing items which are not present in the incoming data. Parameters are the same as /ead with the addition of `ex={id}`, which allows specifying items to ignore, and fondsId, which allows synchronising just within a specific item.

Parameters:

fonds - the ID of a specific fonds within the repository scope to synchronise. If missing the scope will be used

ex - the ID of an item to be excluded from the sync operation

Returns:

a SyncLog instance.

Throws:

ItemNotFound

ValidationError

IOException

DeserializationError

importEag

public ImportLog importEag(String scopeId,
                           Boolean tolerant,
                           Boolean allowUpdates,
                           String logMessage,
                           String propertyFile,
                           String handlerClass,
                           String importerClass,
                           boolean commit,
                           InputStream data)
                    throws ItemNotFound,
                           ValidationError,
                           IOException,
                           DeserializationError

Import EAG files. See EAD import for details.

Throws:

ItemNotFound

ValidationError

IOException

DeserializationError

importEac

public ImportLog importEac(String scopeId,
                           Boolean tolerant,
                           Boolean allowUpdates,
                           String logMessage,
                           String propertyFile,
                           String handlerClass,
                           String importerClass,
                           boolean commit,
                           InputStream data)
                    throws ItemNotFound,
                           ValidationError,
                           IOException,
                           DeserializationError

Import EAC files. See EAD import for details.

Throws:

ItemNotFound

ValidationError

IOException

DeserializationError

importCsv

public ImportLog importCsv(String scopeId,
                           Boolean tolerant,
                           Boolean allowUpdates,
                           String logMessage,
                           String importerClass,
                           boolean commit,
                           InputStream data)
                    throws ItemNotFound,
                           ValidationError,
                           IOException,
                           DeserializationError

Import a set of CSV files. See EAD handler for options and defaults but substitute text/csv for the input mimetype when a single file is POSTed.

Additional note: no handler class is required.

Throws:

ItemNotFound

ValidationError

IOException

DeserializationError

importLinks

public ImportLog importLinks(Boolean tolerant,
                             boolean commit,
                             Table table)
                      throws DeserializationError,
                             ItemNotFound

Create multiple links via CSV or JSON tabular upload.

Each data row must consist of 5 columns:

1. the source item
2. the target item
3. the link type, e.g. associative, hierarchical
4. the link field (if applicable) e.g. relatedUnitsOfDescription
5. the link description

A separate log item will be created for each row.

Parameters:

table - the tabular data

Returns:

a single column table of link IDs

Throws:

DeserializationError - the problems are found with the import data

ItemNotFound