Submitting Files to an Archive and Checking File Formats

There are two methods you can use with your web browser to submit files to an archive. These can also be used to just check file formats without archiving them. Both operations require a valid archive account name and login password, which are the same you would use when logging into an archive account via ftp. Contact the curator of this site if you need the login information for an archive account.

When submitting files to an archive you must also provide the PI password associated with your files, which means that your files must already be registered with the archivist and included in the archive database. See Before You Submit Files... for more details.

If you just want to check data file formats then it is best to use valid file names, as defined in filenames.txt. If you use arbitrary file names then consistency checks between file names and data file header information cannot be performed.

When you submit data files, or just check their format, the data file formats are thoroughly checked for consistency with the format specifications in File Formats. The rate at which formats are checked is approximately 12 MB/minute. When the operation is completed you will be given the option to view the resulting diagnostics, and I recommend that you review the diagnostics to ensure the operation produced the desired results. The notation in the diagnostics mimics the notation in the format specs document when referring to independent and dependent variables. Diagnostics preceded by one asterisk (*) indicate a notice or warning; those preceded by two or three asterisks (** or ***) indicate errors which terminate the check/submit operation.

The check/submit operations will modify your files in the following ways:

• Macintosh and DOS text files are converted to Unix text files.
• Horizontal TAB characters in ASCII data files are converted to SPACE characters.
• A LF character is added to the last line of improperly terminated text files.
• All lower case characters in file names are converted to upper case.

Upload and Submit or Check One File

This operation allows you to upload and either check or submit one file at a time. All you need for this operation is a web browser which knows how to upload files.

Submit or Check Several Files

To use this operation you must first use an ftp client program to upload your files to the files directory of the appropriate archive account. The following example illustrates some of the available commands you can use to upload both ASCII data files and binary image files, using a command line interface to the ftp program. This example uses the solve2 archive account, with descriptive annotations for the commands enclosed in curly braces, {}.

  ftp espoarchive.nasa.gov       {initiate ftp connection}
  Name (): solve2                {supply archive account name}
  Password:                      {enter the login password}
  ftp> cd files                  {change directory to files}
  ftp> ascii                     {set ASCII transfer mode}
  ftp> mput dd*.dc8              {upload DD.DC8 ASCII data files}
  ftp> dir                       {list the directory contents}
  ftp> binary                    {set binary transfer mode}
  ftp> mput ii*.png              {upload II.PNG binary image files}
  ftp> dir
  ftp> quit                      {logout}

After your files are in the appropriate files directory, you can use the Submit or Check Several Files operation to either check or submit your files. Or, you can first check them and then submit them.

Because ASCII data file formats are checked at the approximate rate of 12 MB/minute, and the web server timeout for send/receive operations is 5 minutes, a notable limitation of this operation is that you may not be able to view the diagnostics from this operation if the cumulative size of the data files being processed is too large. You can work around this limitation by using the web form several times and judiciously specifying the files to process each time you submit the form.

For example, suppose the cumulative size of all of the DD.DC8 files in the above ftp example is 80 MB, and you want to submit them in a way that you will be able to view the diagnostics. Also suppose that by examining the output from the ftp dir command you see that you can partition the files into 10 day time periods (by file names) and end up with approximately 40 MB for the files in each partition. So you can fill out and submit the web form for this operation twice and each time specify different files to process:

Enter file name spec:   dd2003011*.dc8      {first time}
Enter file name spec:   dd2003012*.dc8      {second time}

Before You Submit Files...

The archive database must include the appropriate information about your files (data or image files) before they can be archived. The required information for each type of file you intend to submit is:

1. A file name.
   Each archived file must have a unique name. As described in filenames.txt, each type of file can be conveniently represented by a file name code, which is constructed from the constant parts of the file name. The archive database must include the file name code for your files, and from that it can recognize your files.
2. The Point of Contact (POC) for the file.
   This can be the Principal Investigator (PI) for the group producing the files, or some other person in that group who should be contacted in case someone has questions about the file.
3. A brief description of the contents or subject of the file.
4. A PI password for the file.
   This password is only required when you submit files to an archive. Its only function is to ensure that someone else does not accidentally submit a file with the same name as your file, and thus replace your file.

Please include that information in an email message to the curator of this site so it can be entered in the archive database. If you are unsure about how best to name your files, just send an email stating your intentions and desires, and a naming convention will be resolved iteratively.

You can verify whether or not your files are registered by examining the file catalogs, datatable.html and imagetable.html, which reside in the docs directory of the appropriate archive account. If your file codes are included in a file catalog then you can submit your files.

The Traditional ftp/telnet Method

The traditional method for submitting files to an archive, using ftp and telnet/ssh, has been abandoned. Please use one of the two methods described above to check data file formats and submit files to an archive.

Curator: Steven Gaines  -  NASA Official: Mike Craig
Updated: 2005-01-29