\(\renewcommand\AA{\text{Å}}\)
19. GSAS-II Import Modules
Imports are implemented by deriving a class from
GSASIIobj.ImportPhase, GSASIIobj.ImportStructFactor,
GSASIIobj.ImportPowderData ,
GSASIIobj.ImportSmallAngleData,
GSASIIobj.ImportReflectometryData,
GSASIIobj.ImportPDFData,
or GSASIIobj.ImportImage. These classes are in turn
derived from GSASIIobj.ImportBaseclass.
Module file names (G2phase_, G2pwd_ and G2sfact_, etc.) are used to
determine which menu an import routine should be placed into. (N.B. this
naming was an unnecessary choice; importer types could have been determined
from the base class.)
To implement import of
a phase, a single crystal or a powder dataset, respectively, name the
file with the appropriate file name and place the file anywhere in the
path defined in sys.path and the next time GSAS-II is started,
the file should be read by Python and the new format will appear in
the appropriate import menu.
Importers are documented below, separated by type. Importers tend to
be fairly simple files, where many are in the range of 50-100 lines,
and where more than half of those lines are directly copied from other
importers without any changes. Details on this are given in the
Writing a Import Routine section, below.
19.1. Phase Import Routines
Phase import routines are classes derived from
GSASIIobj.ImportPhase.
They must be found in files named G2phase*.py that are in the Python path
and the class must override the __init__ method and add a Reader method.
The distributed routines are:
19.1.1. Module G2phase: PDB, .EXP & JANA m40,m50
A set of short routines to read in phases using routines that were previously implemented in GSAS-II: PDB, GSAS .EXP and JANA m40-m50 file formats
- class G2phase.EXP_ReaderClass[source]
Routine to import Phase information from GSAS .EXP files
- Reader(filename, ParentFrame=None, usedRanIdList=[], **unused)[source]
Read a phase from a GSAS .EXP file using
ReadEXPPhase()
- class G2phase.JANA_ReaderClass[source]
Routine to import Phase information from a JANA2006 file
- ContentsValidator(filename)[source]
Taking a stab a validating a .m50 file (look for cell & at least one atom)
- Reader(filename, ParentFrame=None, **unused)[source]
Read a m50 file using
ReadJANAPhase()
- class G2phase.PDB_ReaderClass[source]
Routine to import Phase information from a PDB file
- ContentsValidator(filename)[source]
Taking a stab a validating a PDB file (look for cell & at least one atom)
- Reader(filename, ParentFrame=None, **unused)[source]
Read a PDF file using
ReadPDBPhase()
- class G2phase.PDF_ReaderClass[source]
Routine to import Phase information from ICDD PDF Card files
- Reader(filename, ParentFrame=None, **unused)[source]
Read phase from a ICDD .str file using
ReadPDFPhase()
19.1.2. Module G2phase_GPX: Import phase from GSAS-II project
Copies a phase from another GSAS-II project file into the current project.
19.1.3. Module G2phase_CIF: Coordinates from CIF
Parses a CIF using PyCifRW from James Hester (https://github.com/jamesrhester/pycifrw) and pulls out the structural information.
If a CIF generated by ISODISTORT is encountered, extra information is added to the phase entry and constraints are generated.
- class G2phase_CIF.CIFPhaseReader[source]
Implements a phase importer from a possibly multi-block CIF file
- ContentsValidator(filename)[source]
This routine will attempt to determine if the file can be read with the current format. This will typically be overridden with a method that takes a quick scan of [some of] the file contents to do a “sanity” check if the file appears to match the selected format. the file must be opened here with the correct format (binary/text)
- ISODISTORT_proc(blk, atomlbllist, ranIdlookup, filename)[source]
Process ISODISTORT items to create constraints etc. Constraints are generated from information extracted from loops beginning with _iso_ and are placed into self.Constraints, which contains a list of constraints tree items and one dict. The dict contains help text for each generated ISODISTORT variable
At present only _iso_displacivemode… and _iso_occupancymode… are processed. Not yet processed: _iso_magneticmode…, _iso_rotationalmode… & _iso_strainmode…
19.1.4. Module G2phase_INS: Import phase from SHELX INS file
Copies a phase from SHELX ins file into the current project.
- class G2phase_INS.PhaseReaderClass[source]
Opens a .INS file and pulls out a selected phase
- Reader(filename, filepointer, ParentFrame=None, **unused)[source]
Read a ins file using
ReadINSPhase()
19.1.5. Module G2phase_rmc6f: Import phase from RMCProfile
Copies a phase from a file written by RMCProfile into the current GSAS-II project.
19.1.6. Module G2phase_xyz: read coordinates from an xyz file
A short routine to read in a phase from an xyz Cartesian coordinate file
19.2. Powder Data Import Routines
Powder data import routines are classes derived from
GSASIIobj.ImportPowderData.
They must be found in files named G2pwd*.py that are in the Python path
and the class must override the __init__ method and add a
Reader method.
The distributed powder data importers are:
19.2.1. Module G2pwd_GPX: GSAS-II projects
Routine to import powder data from GSAS-II .gpx files
- class G2pwd_GPX.GSAS2_ReaderClass[source]
Routines to import powder data from a GSAS-II file This should work to pull data out from a out of date .GPX file as long as the details of the histogram data itself don’t change
19.2.2. Module G2pwd_fxye: GSAS data files
Routine to read in powder data in a variety of formats that were defined in the original GSAS/EXPGUI software suite.
- class G2pwd_fxye.GSAS_ReaderClass[source]
Routines to import powder data from a GSAS files
- ContentsValidator(filename)[source]
Validate by checking to see if the file has BANK lines & count them
- Reader(filename, ParentFrame=None, **kwarg)[source]
Read a GSAS (old formats) file of type FXY, FXYE, ESD or STD types. If multiple datasets are requested, use self.repeat and buffer caching.
EDS data is only in the STD format (10 values per line separated by spaces); the 1st line contains at col 60 the word “Two-Theta “ followed by the appropriate value. The BANK record contains the 3 values (4th not used) after ‘EDS’ for converting MCA channel number (c) to keV via E = A + Bc + Cc^2; these coefficients are generally predetermined by calibration of the MCA. They & 2-theta are transferred to the Instrument parameters data.
19.2.3. Module G2pwd_xye: Topas & Fit2D data
Routine to read in powder data from a number of related formats including ones used in Topas and Fit2D. Typical file extensions are .xye, .qye, .chi, and .qchi.
19.2.4. Module G2pwd_CIF: CIF powder data
Routine to read in powder data from a CIF. Parses a CIF using PyCifRW from James Hester (https://github.com/jamesrhester/pycifrw).
19.2.5. Module G2pwd_BrukerRAW: Bruker .raw & .brml
Routine to read in powder data from a Bruker versions 1, 2, or 3 .raw or a .brml file. Alas, we have not been able to get documentation for the version 4 .raw file, so this is not yet supported.
- class G2pwd_BrukerRAW.brml_ReaderClass[source]
Routines to import powder data from a zip Bruker .brml file
- ContentsValidator(filename)[source]
This routine will attempt to determine if the file can be read with the current format. This will typically be overridden with a method that takes a quick scan of [some of] the file contents to do a “sanity” check if the file appears to match the selected format. the file must be opened here with the correct format (binary/text)
19.2.6. Module G2pwd_FP: FullProf .dat data
Routine to read in powder data from a FullProf .dat file
19.2.7. Module G2pwd_Panalytical: Panalytical .xrdml data
Routines to import powder data from a Pananalytical (XML) .xrdm file.
- class G2pwd_Panalytical.Panalytical_ReaderClass[source]
Routines to import powder data from a Pananalytical.xrdm (xml) file.
- ContentsValidator(filename)[source]
This routine will attempt to determine if the file can be read with the current format. This will typically be overridden with a method that takes a quick scan of [some of] the file contents to do a “sanity” check if the file appears to match the selected format. the file must be opened here with the correct format (binary/text)
19.2.8. Module G2pwd_csv: Read Excel .csv data
Routine to read in powder data from Excel type comma separated variable column-oriented variable. The only allowed extensions for this are .csv, .xy, or .XY.
- class G2pwd_csv.csv_ReaderClass[source]
Routines to import powder data from a .xye file
- ContentsValidator(filename)[source]
This routine will attempt to determine if the file can be read with the current format. This will typically be overridden with a method that takes a quick scan of [some of] the file contents to do a “sanity” check if the file appears to match the selected format. the file must be opened here with the correct format (binary/text)
19.2.9. Module G2pwd_rigaku: powder data from a Rigaku .txt file
- class G2pwd_rigaku.Rigaku_rasReaderClass[source]
Routines to import powder data from a Rigaku .ras file with multiple scans. All scans will be imported as individual PWDR entries
- ContentsValidator(filename)[source]
This routine will attempt to determine if the file can be read with the current format. This will typically be overridden with a method that takes a quick scan of [some of] the file contents to do a “sanity” check if the file appears to match the selected format. the file must be opened here with the correct format (binary/text)
- class G2pwd_rigaku.Rigaku_txtReaderClass[source]
Routines to import powder data from a Rigaku .txt file with an angle and then 1 or 11(!) intensity values on the line. The example file is proceeded with 10 of blank lines, but I have assumed they could be any sort of text. This code should work with an angle and any number of intensity values/line as long as the number is the same on each line. The step size may not change. The number of comment lines can also change, but should not appear to be intensity values (numbers only).
- ContentsValidator(filename)[source]
This routine will attempt to determine if the file can be read with the current format. This will typically be overridden with a method that takes a quick scan of [some of] the file contents to do a “sanity” check if the file appears to match the selected format. the file must be opened here with the correct format (binary/text)
19.3. Single Crystal Data Import Routines
Single crystal data import routines are classes derived from
, GSASIIobj.ImportStructFactor.
They must be found in files named G2sfact*.py that are in the Python path
and the class must override the __init__ method and add a Reader method.
The distributed routines are:
19.3.1. Module G2sfact: simple HKL import
Read structure factors from a number of hkl file types. Routines are provided to read from files containing F or F2 values from a number of sources.
- G2sfact.ColumnValidator(parent, filepointer, nCol=5)[source]
Validate a file to check that it contains columns of numbers
- class G2sfact.M90_ReaderClass[source]
Routines to import F**2, sig(F**2) reflections from a JANA M90 file
- class G2sfact.NIST_hb3a_INT_ReaderClass[source]
Routines to import neutron CW F**2, sig(F**2) reflections from a NIST hb3a int file
- class G2sfact.NT_HKLF2_ReaderClass[source]
Routines to import neutron TOF F**2, sig(F**2) reflections from a HKLF file
- class G2sfact.NT_JANA2K_ReaderClass[source]
Routines to import neutron TOF F**2, sig(F**2) reflections from a JANA2000 file
- class G2sfact.SHELX4_ReaderClass[source]
Routines to import F**2, sig(F**2) reflections from a Shelx HKLF 4 file
- class G2sfact.SHELX5_ReaderClass[source]
Routines to import F**2, sig(F**2) twin/incommensurate reflections from a fixed format SHELX HKLF5 file
19.3.2. Module G2sfact_CIF: CIF import
Read structure factors from a CIF reflection table CIF using PyCifRW from James Hester (https://github.com/jamesrhester/pycifrw).
19.4. Small Angle Scattering Data Import Routines
Small angle scattering data import routines are classes derived from
, GSASIIobj.ImportSmallAngle.
They must be found in files named G2sad*.py that are in the Python path
and the class must override the __init__ method and add a Reader method.
The distributed routines are in:
19.4.1. Module G2sad_xye: read small angle data
Routines to read in small angle data from an .xye type file, with two-theta or Q steps. Expected extensions are .xsad, .xdat, .nsad, or .ndat.
- class G2sad_xye.txt_NeutronReaderClass[source]
Routines to import neutron q SAXD data from a .nsad or .ndat file
- class G2sad_xye.txt_XRayReaderClass[source]
Routines to import X-ray q SAXD data from a .xsad or .xdat file
19.5. Image Import Routines
Image import routines are classes derived from
GSASIIobj.ImportImage.
See Writing a Import Routine for general
information on importers and the GSASIIobj.ImportImage for
information on what class variables a reader should set.
Image importers must be found in files named G2img*.py that are in the Python path
and the class must override the __init__ method and add a
Reader method.
The distributed routines are:
19.5.1. Module G2img_ADSC: .img image file
19.5.2. Module G2img_EDF: .edf image file
19.5.3. Module G2img_SumG2: Python pickled image
Routine to read an image from GSAS-II that has been pickled in Python. Images in this format are created by the “Sum image data” command. At least for now, only one image is permitted per file.
19.5.4. Module G2img_GE: summed GE image file
Read data from General Electric angiography x-ray detectors, primarily as used at APS 1-ID. This shows an example of an importer that will handle files with more than a single image.
- class G2img_GE.GE_ReaderClass[source]
Routine to read a GE image, typically from APS Sector 1.
The image files may be of form .geX (where X is ‘ ‘, 1, 2, 3, 4 or 5), which is a raw image from the detector. These files may contain more than one image and have a rudimentary header. Files with extension .sum or .cor are 4 byte integers/pixel, one image/file. Files with extension .avg are 2 byte integers/pixel, one image/file.
- Reader(filename, ParentFrame=None, **kwarg)[source]
Read using GE file reader,
GetGEsumData()
- class G2img_GE.GEsum_ReaderClass[source]
Routine to read multiple GE images & sum them, typically from APS Sector 1.
The image files may be of form .geX (where X is ‘ ‘, 1, 2, 3, 4 or 5), which is a raw image from the detector. These files may contain more than one image and have a rudimentary header. Files with extension .sum or .cor are 4 byte integers/pixel, one image/file. Files with extension .avg are 2 byte integers/pixel, one image/file.
- Reader(filename, ParentFrame=None, **kwarg)[source]
Read using GE file reader,
GetGEsumData()
19.5.5. Module G2img_MAR: MAR image files
19.5.6. Module G2img_Rigaku: .stl image file
19.5.7. Module G2img_1TIF: Tagged-image File images
Routine to read an image in Tagged-image file (TIF) format as well as a variety of slightly incorrect pseudo-TIF formats used at instruments around the world. This uses a custom reader that attempts to determine the instrument and detector parameters from various aspects of the file, not always successfully alas.
Note that the name G2img_1TIF is used so that this file will
sort to the top of the image formats and thus show up first in the menu.
(It is the most common, alas).
- G2img_1TIF.GetTifData(filename)[source]
Read an image in a pseudo-tif format, as produced by a wide variety of software, almost always incorrectly in some way.
- class G2img_1TIF.TIF_ReaderClass[source]
Reads TIF files using a routine (
GetTifData()) that looks for files that can be identified from known instruments and will correct for slightly incorrect TIF usage.- Reader(filename, ParentFrame=None, **unused)[source]
Read the TIF file using
GetTifData()which attempts to recognize the detector type and set various parameters
19.5.8. Module G2img_PILTIF: Std Tagged-image File images
Routine to read an image in Tagged-image file (TIF) format using a standard image library function in Pillow or the now obsolete PIL package. This means that parameters such as the pixel size (which is in the TIFF header but is almost never correct) and distance to sample, etc. are not correct unless specified in a separate metadata file. See below for more information on metadata files.
The metadata can be specified in a file with the same name and path as the TIFF file except that the the extension is .metadata.
The contents of that file are a series of lines of form:
keyword = value
Note that capitalization of keywords is ignored. Defined keywords are in table below. Any line without one of these keywords will be ignored.
keyword |
explanation |
|---|---|
wavelength |
Wavelength in \(\AA\) |
distance |
Distance to sample in mm |
polarization |
Percentage polarized in horizontal plane |
sampleChangerCoordinate |
Used for sample changers to track sample |
pixelSizeX |
Pixel size in X direction (microns) |
pixelSizeY |
Pixel size in Y direction (microns) |
CenterPixelX |
Location of beam center as a pixel number (in X) |
CenterPixelY |
Location of beam center as a pixel number (in X) |
19.5.9. Module G2img_png: png image file
Routine to read an image in .png (Portable Network Graphics) format. For now, the only known use of this is with converted Mars Rover (CheMin) tif files, so default parameters are for that.
19.5.10. Module G2img_CBF: .cbf cif image file
- class G2img_CBF.CBF_ReaderClass[source]
Routine to read a Read cif image data .cbf file. This is used by Pilatus.
- Reader(filename, ParentFrame=None, **unused)[source]
Read using Bob’s routine
GetCbfData()
19.5.11. Module G2img_HDF5: summed HDF5 image file
Reads images found in a HDF5 file. If the file contains multiple images, all are read.
- class G2img_HDF5.HDF5_Reader[source]
Routine to read a HD5 image, typically from APS Sector 6. B. Frosik/SDM.
- ContentsValidator(filename)[source]
Test if valid by seeing if the HDF5 library recognizes the file.
- Reader(filename, ParentFrame=None, **kwarg)[source]
Scan file structure using
visit()and map out locations of image(s) then read one image usingreadDataset(). Save map of file structure in buffer arg, if used.
19.5.12. Module G2img_SFRM: Brucker .sfrm image file
- G2img_SFRM.GetGFRMData(self, filename)[source]
Read Bruker compressed binary detector data gfrm file
- class G2img_SFRM.SFRM_ReaderClass[source]
Routine to read a Read Bruker Advance image data .sfrm/.grfm file.
- ContentsValidator(filename)[source]
GFRM files always begin with FORMAT, should also contain VERSION (also HDRBLKS, but not checked) No check for SRFM files
- Reader(filename, ParentFrame=None, **unused)[source]
Read using Bob’s routine
GetSFRMData()
19.6. PDF Import Routines
PDF import routines are classes derived from
GSASIIobj.ImportPDFData.
See Writing a Import Routine for general information on importers.
The distributed routines are in:
19.6.1. Module G2pdf_gr: read PDF G(R) data
Routines to read in G(R) data from a pdfGet/GSAS-II .gr or gudrun .dat file (with \(\AA\) steps) or S(Q) data from a .fq file.
19.7. Reflectometry Import Routines
Reflectometry import routines are classes derived from
GSASIIobj.ImportReflectometryData.
See Writing a Import Routine for general information on importers.
The distributed routines are:
19.7.1. Module G2rfd_xye: read reflectometry data
Routines to read in reflectometry data from an .xrfd, .xdat, .xtrfd, .xtdat, .nrfd or .ndat type file, with two-theta or Q steps.
- class G2rfd_xye.txt_NeutronReaderClass[source]
Routines to import neutron q REFD data from a .nrfd or .ndat file
19.7.2. Module G2rfd_Panalytical: read Panalytical reflectometry data
Routine to import reflectivity data from a Panalytical .xrdm (xml) file.
- class G2rfd_Panalytical.Panalytical_ReaderClass[source]
Routines to import reflectivity data from a Panalytical.xrdm (xml) file.
- ContentsValidator(filename)[source]
This routine will attempt to determine if the file can be read with the current format. This will typically be overridden with a method that takes a quick scan of [some of] the file contents to do a “sanity” check if the file appears to match the selected format. the file must be opened here with the correct format (binary/text)
19.8. Writing an Import Routine
When writing a import routine, one should create a new class derived
from
GSASIIobj.ImportPhase, GSASIIobj.ImportStructFactor,
GSASIIobj.ImportPowderData ,
GSASIIobj.ImportSmallAngleData,
GSASIIobj.ImportReflectometryData,
GSASIIobj.ImportPDFData,
or GSASIIobj.ImportImage. As described below,
all these classes will implement
an __init__() and a Reader() method, and most will supply a
ContentsValidator() method, too.
See the appropriate class documentation
for details on what values each type of Reader() should
set. General principles on how an importer works are described below.
19.8.1. __init__()
The __init__ method will follow standard boilerplate:
def __init__(self):
super(self.__class__,self).__init__( # fancy way to self-reference
extensionlist=('.ext1','ext2'),
strictExtension=True,
formatName = 'example image',
longFormatName = 'A longer description that this is an example image format'
)
The first line in the __init__ method calls the parent class
__init__ method with the following parameters:
extensionlist: a list of extensions that may be used for this type of file.
strictExtension: Should be True if only files with extensions inextensionlistare allows; False if all file types should be offered in the file browser. Also if False, the import class will be used on all files when “guess from format” is tried, though readers with matching extensions will be tried first. It is a very good idea to supply a ContentsValidator method whenstrictExtensionis False.
formatName: a string to be used in the menu. Should be short.
longFormatName: a longer string to be used to describe the format in help.
Note that if an importer detects a condition which prevents its use,
for example because a required Python package is not present, it can
set the value of self.UseReader to False. Another possible use for
this would be an importer that requires a network connection to a
remote site. Setting self.UseReader to False must be done in the
__init__ method and will prevent the
importer from being used or included in the expected menu.
19.8.2. Reader()
The class must supply a Reader method that actually performs the
reading. All readers must have at a minimum these arguments:
def Reader(self, filename, filepointer, ParentFrame, **unused):
where the arguments have the following uses:
filename: a string with the name of the file being read
filepointer: a file object (created byopen()) that accesses the file and is points to the beginning of the file when Reader is called.
ParentFrame: a reference to the main GSAS-II (tree) windows, for the unusualReaderroutines that will create GUI windows to ask questions. The Reader should do something reasonable such as take a reasonable default ifParentFrameis None, which indicates that GUI should not be accessed.
In addition, the following keyword parameters are defined that Reader
routines may optionally use:
buffer: a dict that can be used to retain information between repeated calls of the routine
blocknum: counts the number of times that a reader is called, to be used with files that contain more than one set of data (e.g. GSAS .gsa/.fxye files with multiple banks or image files with multiple images.)
usedRanIdList: a list of previously used random Id values that can be checked to determine that a value is unique.
As an example, the buffer dict is used in CIF reading to hold the parsed CIF file
so that it can be reused without having to reread the file from
scratch.
19.8.2.1. Reader return values
The Reader routine should return the value of True if the file has been
read successfully. Optionally, use self.warnings to indicate any
problems.
If the file cannot be read, the Reader routine should
return False or raise an GSASIIobj.ImportBaseclass.ImportException
exception. (Why either? Sometimes an exception is the easiest way to
bail out of a called routine.) Place text in self.errors and/or use:
ImportException('Error message')
to give the user information on what went wrong during the reading. The following variables are used to indicate results from the reader:
self.warnings
Use self.warnings to indicate any information that should be displayed to the user if the file is read successfully, but perhaps not completely or additional settings will need to be made.
self.errors
Use self.errors to give the user information on where and why a read
error occurs in the file. Note that text supplied with the raise
statement will be appended to self.errors.
self.repeat
Set self.repeat to True (the default is False) if a Reader should be called again to after reading to indicate that more data may exist in the file to be read. This is used for reading multiple powder histograms or multiple images from a single file. Variable self.repeatcount is used to keep track of the block numbers.
19.8.2.2. Reader support routines
Note that GSASIIIO supplies three routines,
BlockSelector()
MultipleBlockSelector() and
MultipleChoiceSelector() that are useful for
selecting amongst one or more datasets (and perhaps phases) or data items for
Reader() routines that may encounter more than one set of information
in a file.
19.8.3. ContentsValidator()
Defining a ContentsValidator method is optional, but is usually a
good idea, particularly if the file extension is not a reliable
identifier for the file type. The intent of this routine is to take a
superficial look at the file to see if it has the expected
characteristics of the expected file type. For example, are there
numbers in the expected places?
This routine is passed a single argument:
filepointer: a file object (created by
open()) that accesses the file and is points to the beginning of the file when ContentsValidator is called.
Note that GSASIIobj.ImportBaseclass.CIFValidator() is a ContentsValidator
for validating CIF files.
19.8.3.1. ContentsValidator return values
The ContentsValidator routine should return the value of True if
the file appears to match the type expected for the class.
If the file cannot be read by this class, the routine should return False. Preferably one will also place text in self.errors to give the user information on what went wrong during the reading.
19.8.4. ReInitialize()
Import classes are substantiated only once and are used as needed.
This means that if something needs to be initialized before the
Reader() will be called to read a new file, the initialization step must be coded. The
ReInitialize() method is provided for this and it is always called
before the ContentsValidator method is called. Use care to call
the parent class ReInitialize() method, if this is overridden.