Purpose

• Prepare data in bulk, review, and upload.
• Migrating away from an existing implementation, and need to seed with existing data.
• Your organization has a separate component where data is collected outside Avni, but you still need this data to be present with field workers using Avni.

Using the Admin app to upload data

The Admin app of the web console has an upload option. Currently, this supports the following. Essentially for each form present in you organisation there is a corresponding upload option in the dropdown, with a sample file.

• Upload subjects
• Upload program enrolments — active, or already exited
• Upload program encounters — scheduled, completed, or cancelled
• Upload encounters — scheduled, completed, or cancelled
• Upload locations
• Upload users and catchments
• Upload metadata zip file downloaded from a different implementation

Sample file

Sample files are available in the interface. Download the file, fill in values and then upload. The file is in a CSV format.
Sample file acts as an up-to-date documentation on the following.

• fields
• whether they are mandatory for upload
• possible values
• format of the value

📘

Since above has already been documented and maintain in the sample file these are not documented here again, please refer to it as a reference documentation.

Mandatory fields in the form

The mandatory fields in the form element are not applicable when uploading the data via CSV files - since we have seen when made mandatory especially for the legacy data, the users are force to upload some junk information (this may be added in the future).

Rules

No rules are run as part of CSV upload. This implies that:

• field values created automatically via form element rules will not get created (such columns are present in the sample hence can be uploaded manually)
• observations created by decision rules will not be created automatically (such columns are present in the sample hence can be uploaded manually)
• Validation rule is not applied
• Edit rule is not applied

📘

Avni currently doesn't have a robust framework to run these rules on the server side. This may be added in future, if we observe that users need these.

Identifiers

The primary purpose of these identifiers is for the users to be able to link different CSV file types upload data to each other - in the same way as foreign key linkages between different records. These linkages can be created using identifiers of user's choosing. Lets try to understand this via an example. Lets assume there are three forms.

• Woman Registration (Subject)
• Pregnancy Program Enrolment (Program Enrolment)
  • links to woman
• Ante Natal Visit Form (Program Encounter)
  • links to pregnancy program

📘

The identifiers used above, for Id from previous system, are saved in Avni but is not visible in Avni after uploading, it is used only for matching records during CSV upload.

Encounter and enrolment upload modes

When the upload type is an encounter, program encounter, or program enrolment, a mode radio group appears below the upload-type dropdown. Each mode has its own sample file, since the required columns differ.

Encounter modes

Available for both general encounters and program encounters:

• Schedule visits — creates a scheduled (not-yet-completed) encounter. Sample includes Earliest Visit Date and Max Visit Date; no visit observations.
• Upload visits — creates a completed encounter. Sample includes Visit Date and the encounter form's observations. (Previously labelled "Upload visit details".)
• Upload cancelled visits — creates an encounter already in the cancelled state. Sample includes Earliest Visit Date, Max Visit Date, Cancel Date, optional Cancel Location, and the encounter's cancellation form observations. There is no Visit Date column. Use this when migrating cancelled encounters from another system (typically during an organisation merge).

Scheduling a visit and then later uploading the visit details for the same visit is not supported — that would be an edit, which CSV upload does not handle.

Program enrolment modes

• Upload enrolments (default) — creates an active program enrolment. Same shape as before.
• Upload exited enrolments — creates a program enrolment that is already in the exited state, from a single row carrying both regular and exit observations. Sample includes Enrolment Date, optional Enrolment Coordinates, the enrolment form's observations, Exit Date, optional Exit Coordinates, and the program-exit form's observations. Use this when migrating exited enrolments from another system (typically during an organisation merge).

Exit: prefix convention

In the Upload exited enrolments sample, every column drawn from the program-exit form is prefixed with the literal string Exit: to disambiguate it from enrolment-form columns (the same concept may appear on both forms).

• Valid: Exit: Weight at exit, Exit: Vitals at exit|Weight, Exit: Medications stopped|Drug name|1.
• The prefix is recognised only at the start of the column header. A header containing Exit: anywhere else (e.g. Vitals|Exit: Weight) is rejected.

On import, Exit:-prefixed columns are stored as exit observations; remaining concept columns are stored as regular observations.

Important Notes / Gotchas

• Limited Concept Support in CSV Upload: Not all concepts are supported when uploading data via CSV. Specifically, the following are not supported:
  • GroupAffiliation
  • Id
  • File
• Id Confusion: The identifiers (used in Id from previous system) are different from Id elements in the form, if you have them.
• Form Data Editing: Editing previously submitted form data is not currently supported through the CSV upload process.
• No Metadata Rollback: For metadata zip uploads, uploading an older zip file version does not restore a previous version of the metadata. The result of uploading an older version will be an attempted merge of the old version over the current version which might not be what is expected. Avni does not support rolling back of the metadata configuration to a previous state. One can only move forward from the current configuration.
• Multi-select Coded Concepts: For multi-select answers, expected input format would be:
  1. Answer 1, Answer 2, ...
  2. Answer 1, "Answer2, has, commas", Answer 3, ...
  3. "Single answer, with commas" or "Single answer, with commas"
  4. CSV-style escaped quotes: """Answer, with, commas""" ... etc.

Questions

What if I have a comma in my observation value?

• Wrap your value in quotes.

Why are decision concepts not appearing the sample file

If you are using decision concepts in the rule but not linked those concepts then this will happen.

Is the order of values important?

• No. Columns can be in any order.

How do I upload images?

• For images, use a url that the avni server can download. Ensure that
  • The images are a direct download link (not a redirect to a page that uses javascript to download). Starting from v16.13, Avni will follow 5 redirects and attempt to download the file to be able to support google drive share links.
  • The image urls end with the image type. eg: https://somedomain.com/images/abc.png