GDX2DAT and GDXF90: Tools for Reading and Writing GDX Files

`GDX2DAT` and `GDXF90`: Tools for Reading and Writing GDX Files

Thomas F. Rutherford*

Department of Economics
University of Colorado

January, 2004

* This research supported by the GAMS Applied General Equilibrium Research Fund. The author remains responsible for any bugs which exist in this software. This software is not officially supported by GAMS Corporation.

GDX2DAT.EXE is a console application based on GDXF90, a Fortran-90 module implemented in Lahey Fortran 95, version 5.7. GDX2DAT.EXE is similar to the GAMS system utility GDXDUMP.EXE which translated binary GDX data files into a flat (text) file format. The purpose of GDX2DAT is to facilitate the translation of GAMS data for conventional programming and modelling languages, particularly those, such as GEMPACK, in which symbol sequencing is important.

The program operates from the command line as follows:


	gdx2dat GDX file [-s] [item list] [>output file]

The first argument to the program is the GDX filename prefix. The input file is GDX file.gdx, in which the suffix is appended by the program.

The second argument is an optional switch indicating that external set definitions are to be read. When the -s switch is applied, the program then reads set definitions from GDX file.set. [item list] consists of sets and/or parameters from the GDX file. The program does not yet support equations and multidimensional sets. If [item list] is omitted, the program processes all the sets and parameters in the GDX file.

Program output is directed to the console, but output may be written to a text file using output redirection on the invocation.

Generic Invocation of `GDX2DAT.EXE`

In order to illustrate the program's operation, consider the following GAMS program:

$title	Produces DEMO.GDX for illustrating use of GDX2DAT program

set	i	Ordered set /i1*i10/,
	j(i)	Subset of i /i2*i6/,,
	k(i,j)	A tuple defined on i and j;

k(i,j)$(uniform(0,1) gt 0.5) = yes;
display k;

parameters
	s	Scalar /0.63/,
	v(i)	Vector
	m(i,j)	Matrix;

variable
	x(i)	Variable;

*	Generate a sparse vector and a sparse matrix:

v(i)$(uniform(0,1) le 0.5) = uniform(0,1);
m(i,j)$(uniform(0,1) le 0.25) = (ord(i)-1)*card(j) + card(j);

display s, v, m;

x.l(i) = ord(i);
x.m(i) = 10 * ord(i);
x.lo(i) = 100 * ord(i);
x.up(i) = 1000 * ord(i);
x.scale(i) = 10000 * ord(i);

If DEMO.GMS is processed with the command:


gams demo gdx=demo

then the listing file, DEMO.LST, contains the following output:


----      8 SET k  A tuple defined on i and j

             i2          i3          i4          i5          i6

i1                      YES         YES
i2                                  YES                     YES
i3          YES         YES         YES         YES
i4          YES                                 YES
i5                                                          YES
i6          YES                     YES         YES
i7                      YES                     YES
i8                      YES         YES         YES
i10                     YES         YES         YES


----     23 PARAMETER s                    =        0.630  Scalar

----     23 PARAMETER v  Vector

i4 0.086,    i5 0.641,    i7 0.792,    i8 0.176


----     23 PARAMETER m  Matrix

             i2          i3          i4          i5          i6

i1        5.000       5.000
i2                   10.000      10.000      10.000
i3                                           15.000
i4       20.000      20.000      20.000
i5       25.000      25.000
i7                               35.000                  35.000
i8                   40.000      40.000
i9                   45.000                              45.000
i10                              50.000

If, the GDX file is processed using the command:


gdx2dat demo >demo.dat

then the output file is written to the the DEMO.DAT text file. The output syntax is designed to be easily read by a programming language such as C, Fortran, Java or Delphi.

GDX2DAT processes both single dimensional sets and tuples. The syntax for sets is:


SET ident
n elements
1 element 1
2 element 2
..
n element n

The syntax for tuples is


TUPLE ident
m dimensions
n elements
i1 i2 ... im	(for element 1)
i1 i2 ... im	(for element 2)
...
i1 i2 ... im	(for element n)

The syntax for parameters is:


PARAMETER ident
m dimensions
n nonzeros
i1 i2 .. im  value1
i1 i2 .. im  value2
i1 i2 .. im  value3
...
i1 i2 .. im  valuen

The output begins with a report of the GDX file contents:

* Reading GDX file: demo.gdx
* Symbols: 7
* UELS: 10
* SET i ndim=1 nele=10 Ordered set
* SET j ndim=1 nele=5 Subset of i
* SET k ndim=2 nele=22 A tuple defined on i and j
* PARAMETER s ndim=0 nele=1 Scalar
* PARAMETER v ndim=1 nele=4 Vector
* PARAMETER m ndim=2 nele=18 Matrix
* VARIABLE x ndim=1 nele=10 Variable

There are two one-dimensional sets in DEMO.GDX. The smaller of these is set j which is output as follows:


SET j
5 elements
1 i2 
2 i3 
3 i4 
4 i5 
5 i6

There is a single two-dimensional set in DEMO.GDX which is output as follows:


TUPLE k
2 dimensions
22 elements
 1 1 
 1 2 
...
 9 5

Scalar parameters are reported as parameters with 0 dimensions and 1 nonzero. Parameter S from DEMO.GDX produces the following output:


PARAMETER s
0 dimensions
1 nonzeros
 0.6300000000000000

Vectors are reported as parameters with 1 dimension and two or more nonzeros. The nonzeros are reported with integer set indices. Parameter V from DEMO.GMS appears as follows:


PARAMETER v
1 dimension
4 nonzeros
 1 8.642462400000001E-02
 2 0.6412511510000001
 3 0.7923606420000000
 4 0.1756610490000000

Care should be taken in interpreting the indices of sparse parameters. In this example, parameter V is defined on the ordered set I but the parameter has four nonzeros. The association between indices and set labels is provided in the set which appears immediately after the parameter:


SET v(.)
4 elements
1 i4
2 i5
3 i7
4 i8

The set identifier in this case is V(.). The "." in this identifier stands for the elements of set I for which there are nonzero elements in V.

The two dimensional parameter M appears as follows in the GDX2DAT output:


PARAMETER m
2 dimensions
18 nonzeros
 1 1 5.000000000000000
 1 2 5.000000000000000
..

The indices labelling this parameter are reported in two subsequent sets. The first set associates row indices with elements of set I:


SET m(.,)
9 elements
1 i1
2 i2
3 i3
4 i4
5 i5
6 i7
7 i8
8 i9
9 i10

The second set translates column indices to elements of I:


SET m(,.)
5 elements
1 i2
2 i3
3 i4
4 i5
5 i6

Explicit Invocation of `GDX2DAT.EXE`

The program permits considerable control over the indexing of data output. It is also possible to extract sets or parameters from the GDX file one at a time. Here is an example of how the program can be used to extract a single parameter:


gdx2dat demo v(i) >demo.dat

This command generates output in which the indices of elements in V correspond to the sequence of elements in set i, i.e. 4 corresponds to i4, 5 corresponds to i5, etc.:


PARAMETER v(i)
1 dimension
4 nonzeros
 4 8.642462400000001E-02
 5 0.6412511510000001
 7 0.7923606420000000
 8 0.1756610490000000

The GDX2DAT utility can be used to extract subsets of rows or columns from parameters. For example, the command:<


gdx2dat demo m(i,'i4') >demo.dat

produces output:


PARAMETER m(i,'i4')
1 dimension
5 nonzeros
 2 10.00000000000000
 4 20.00000000000000
 7 35.00000000000000
 8 40.00000000000000
 10 50.00000000000000

N.B. Use single quotes when referencing singleton elements. Due to a technical problem with the command shell interface, double quotes on the command line produce the following error message:


Error: double quotes are not permitted on the command line.  Use single quotes.

User-Specified Sets

Indexing of output from GDX2DAT.EXE can be controlled through the provision of user-specified sets. A set-definition file has the GDX filename prefix and a .SET suffix. A set file which can be used in conjunction with DEMO.GDX is DEMO.SET.

This set provides two external sets. The first provides a resequencing of elements from the GDX file, as illustrated by set K which references the first five elements of set I in reverse order:


SET i
5 elements
1 i5
2 i4
3 i3
4 i2
5 i1

An external set may also revise a set definition from the GDX file, as illustrated by set j in DEMO.SET:


SET j
3 elements
1 i4
2 i5
3 i6

When a set is redefined, a warning message is written to the output file header:


* Warning: set i is redefined. 
* Warning: set j is redefined.

External sets can be used to control which data are extracted and how they are labelled. For example, if we issue the command:


gdx2dat demo -s m(i,j)

The output from this command then appears as follows:


PARAMETER m(i,j)
2 dimensions
4 nonzeros
 4 1 10.00000000000000
 4 2 10.00000000000000
 3 2 15.00000000000000
 2 1 20.00000000000000

F90 Interface

GDX2DAT.EXE is based on a Fortran-90 module, GDXF90, which in turn calls Paul van der Eijk's Delphi routines, GDXIO.DLL. (The mixed-language calls involved in GDXF90 require Lahey Fortran 95, version 5.7.) All data to be returned from the GDX file (aside from scalars) must be declared as allocatable. Details of the GDXF90 module are provided here.

DEMO.F90 provides an illustrative program which uses this module. Two routines from GDXF90 are used in DEMO.F90. The first is gdxread, a module component which reads a complete GDX file.

The Fortran program DEMO.F90 illustrates how DEMO.GDX can be read.


program demo
use gdxf90
implicit none
integer :: k,l

!	Set element identifiers are 32 characters in length:

character (len=32), allocatable :: i(:), j(:)

!	Data may be returned in dense format:

real (kind=8) :: s
real (kind=8), allocatable :: v(:), m(:,:)

!	Alternatively, data may be accessed in sparse format:

real (kind=8), allocatable :: md(:)
integer, allocatable :: me(:,:)

!	Read the data file through the GDXREAD routine:

if (.not.gdxread('demo.gdx'))	stop 'Error reading demo.gdx.'

!	Read single-dimensional sets and parameters through the 
!	GDXDATA routine.  

if (.not.gdxdata(i,'i'))		stop 'Error reading set i'
if (.not.gdxdata(j,'j'))		stop 'Error reading set j'
if (.not.gdxdata(s,'s'))		stop 'Error reading parameter s'
if (.not.gdxdata(v,'v(i)'))		stop 'Error reading parameter v(i)'
if (.not.gdxdata(m,'m(i,j)'))		stop 'Error reading parameter m(i,j) (dense)'
if (.not.gdxdata(md,me,'m(i,j)'))	stop 'Error reading parameter m(i,j) (sparse)'

!	Produce an echo-print of data to the console:

write(*,'(//a)') 'SET i:';	do k=1,size(i);  write(*,'(a)') trim(i(k)); end do
write(*,'(//a)') 'SET j:';	do k=1,size(j);  write(*,'(a)') trim(j(k)); end do
write(*,'(//a)') 'SCALAR s:';	write(*,*) s
write(*,'(//a)') 'PARAMETER v:';  do k=1,size(v);  write(*,*) v(k); end do
write(*,'(//a)') 'PARAMETER m(i,j): (dense format)'
do k=1,size(m,1);  write(*,'(10f8.2)') (m(k,l),l=1,size(m,2)); end do
write(*,'(//a)') 'PARAMETER m(i,j): (sparse format)'
do k=1,size(md);  write(*,*) me(k,1),me(k,2), md(k); end do;

end program demo

Program output appears as follows:


* Reading GDX file: demo.gdx
* Symbols: 7
* UELS: 10
* SET i ndim=1 nele=10 Ordered set
* SET j ndim=1 nele=5 Subset of i
* SET k ndim=2 nele=22 A tuple defined on i and j
* PARAMETER s ndim=0 nele=1 Scalar
* PARAMETER v ndim=1 nele=4 Vector
* PARAMETER m ndim=2 nele=18 Matrix
* VARIABLE x ndim=1 nele=10 Variable


SET i:
i1
i2
i3
i4
i5
i6
i7
i8
i9
i10


SET j:
i2
i3
i4
i5
i6


SCALAR s:
 0.6300000000000000


PARAMETER v:
 0.000000000000000E+00
 0.000000000000000E+00
 0.000000000000000E+00
 8.642462400000001E-02
 0.6412511510000001
 0.000000000000000E+00
 0.7923606420000000
 0.1756610490000000
 0.000000000000000E+00
 0.000000000000000E+00


PARAMETER m(i,j): (dense format)
    5.00    5.00    0.00    0.00    0.00
    0.00   10.00   10.00   10.00    0.00
    0.00    0.00    0.00   15.00    0.00
   20.00   20.00   20.00    0.00    0.00
   25.00   25.00    0.00    0.00    0.00
    0.00    0.00    0.00    0.00    0.00
    0.00    0.00   35.00    0.00   35.00
    0.00   40.00   40.00    0.00    0.00
    0.00   45.00    0.00    0.00   45.00
    0.00    0.00   50.00    0.00    0.00


PARAMETER m(i,j): (sparse format)
 1 1 5.000000000000000
 1 2 5.000000000000000
 2 2 10.00000000000000
 2 3 10.00000000000000
 2 4 10.00000000000000
 3 4 15.00000000000000
 4 1 20.00000000000000
 4 2 20.00000000000000
 4 3 20.00000000000000
 5 1 25.00000000000000
 5 2 25.00000000000000
 7 3 35.00000000000000
 7 5 35.00000000000000
 8 2 40.00000000000000
 8 3 40.00000000000000
 9 2 45.00000000000000
 9 5 45.00000000000000
 10 3 50.00000000000000