2 
4 <a href="https://scan.coverity.com/projects/opendap-bes">
5 <img alt="Coverity" src="https://scan.coverity.com/projects/5060/badge.svg"/>
8 README for the OPeNDAP BES
9 ==========================
11 Updated for version 3.19.1
13 BESInterface/XMLInterface improved
15 Testing/Logging Improvements
17 Updated for version 3.19.0
19 New Server function: range() This function returns the min and max
20 values of a variable and, for a vector, a boolean indicating that it
23 GeoTiff file can now be aggregated.
25 See NEWS for more information.
27 Updated for version 3.18.0
31 - hdf5_handler adds support for sinusodial projections.
33 - ncml_handler adds support for the aggregation and metadata
34 manipulation of remotely located datasets. This new capanbility
35 utilizes the exisiting gateway_handler and allows the ncml_handler
36 to aggregate anything the gateway_handler can read.
38 This release fixes a number of bugs:
40 - Improved error logging (as in errors now get logged)
41 https://opendap.atlassian.net/browse/HYRAX-302
43 - Improved server side function response assembly repaired:
44 https://opendap.atlassian.net/browse/HYRAX-352
45 https://opendap.atlassian.net/browse/HYRAX-342
47 - Stopped prepending "nc_" to netCDF file out responses whose
48 download filename begins with a digit:
49 https://opendap.atlassian.net/browse/HYRAX-341
51 - Repaired various BES timeout issues.
52 https://opendap.atlassian.net/browse/HYRAX-341
54 Updated for version 3.17.4
56 Added support for cached metadata objects.
58 Server functions can now return multiple values using specially-named
61 The code now builds using gcc-6.
63 See NEWS for other details.
65 See the INSTALL file for build instructions, including special
66 instructions for building from a (git) cloned repo. This code can
67 build one of two distinct ways and that requires a tiny bit setup.
69 For more information on Hyrax and the BES, please visit our documentation
70 wiki at docs.opendap.org. This will include the latest install and build
71 instructions, the latest configuration information, tutorials, how to
72 develop new modules for the BES, and more.
74 This version has significant performance improvements for aggregations,
75 particularly those that use the JoinNew directive.
78 * DAP2 versus DAP4 support
79 * What's here: What files are in the distribution
80 * About the BES: What exactly is this
81 * Configuration: How to configure the BES
82 * Testing: Once built and configured, how do you know it works?
86 The code on the master branch now supports both DAP2 and DAP4
87 responses, and the data format handlers do as well. Because of that,
88 it must be linked with libdap 3.14, not libdap older versions of
89 libdap (libdap 3.14 contains support for both DAP2 and DAP4, while the
90 older 'libdap' supports only DAP2). If you need to get the DAP2-only
91 version of the software, use the branch named 'dap2'. Please commit
94 Each of the handlers, which can be built as submodules within this
95 code, also has DAP2 and DAP4 support on their master branch and a
96 'dap2' branch for the DAP2-only code.
100 Here there's a bes-config script which will be installing in $prefix/bin that
101 can be used to determine where the libraries and their header files have been
102 installed. This directory also contains some documentation files.
104 _dispatch_: This is where the bulk of the BES source code resides.
105 server: The BES server and standalone executables; build using dispatch
106 cmdln: A command line client which can communicate with the bes server.
108 _dap_: A module that implements the (OPeN)DAP access.
110 _modules_: All the modules that are part of a standard Hyrax distribution.
111 The configure script looks for their dependencies and only tries to build
112 the ones that can be built. Note that the HDF4 and 5 modules are pulled in
113 from separate _git_ repositories.
115 _xmlcommand_: The BES/dispatch software is a framework. The basic commands
116 recognized by it are defined and implemented by software in this directory.
118 _standalone_: The 'standalone version of the server; used for testing.
120 _apache_: An Apache module, currently not part of the default build
122 _ppt_: The PPT implementation. The BES uses PPT for its communication
124 _templates_: A collection of source files which can be used as templates when
125 you write your own handlers/modules, et cetera.
127 _conf_: Where the automake and autoconf configuration files live
129 _docs_: Where some bes documentation resides
133 Note see http://docs.opendap.org/index.php/Hyrax
134 for the latest and most comprehensive documentation.
138 ### Basic Configuration
140 Once the BES software has been installed, you will need to make a few
141 changes to the BES configuration file, which is the bes.conf located in
142 _prefix_`/etc/bes/bes.conf`. Module and handler configuration files will
143 be installed in the _prefix_`/etc/bes/modules` directory.
145 Only a few parameters need to be modified to get the BES up and running
146 for the first time. These parameters are located at the top of the
147 configuration file are are:
149 **BES.ServerAdministrator**
150 - set this to an email address that can be used for users to contact if
151 there are issues with your installation of the server.
153 **BES.User=user_name**
155 **BES.Group=group_name**
156 - set these to a valid username and groupname on your system. We
157 recommend that you create a username and groupname called bes that has
158 permissions to write only to the BES installation directory. We'll
159 need to write to the log file, and that's about it.
161 **BES.LogName=./bes.log**
162 - Set this to the full path and file name for where you want the BES log
165 With this configuration you will be able to start the BES. No handlers
166 or modules have been installed yet, so you won't be able to serve data,
167 but the BES should run.
169 ### Configuration for Hyrax
171 For the BES to run with Hyrax, additional changes will need to be made
172 to the dap.conf and dap-server.conf files, which are located in the
173 modules directory _prefix_`/etc/bes/modules`. The dap module should be
174 installed by default with the BES. If the dap.conf file is not found, be
175 sure that you have libdap installed. The dap-server module, known as the
176 General purpose handlers, adds responses for the DAP ascii response, the
177 DAP info response, and the DAP html form response. You will need to get
178 the dap-server module. You should see a dap-server.conf file in the
181 The changes required for Hyrax are:
183 **BES.Catalog.catalog.RootDirectory** = _path_to_root_data_directory_
184 - Find the BES.Catalog.catalog.RootDirectory parameter in dap.conf. Set
185 this to the root directory of your data. If you're serving data stored
186 in files, this is the place in the file system where those files are
189 **BES.Catalog.catalog.Include**=;
190 - This parameter specifies what files/directories are included in the
191 list of nodes in the catalog. The default is to show everything. After
192 this parameter is looked at, the Exclude parameter is then looked at
193 to see what files you might want to exclude.
195 __BES.Catalog.catalog.Exclude=^\..*;__
196 - This parameter specifies what files/directories to include in the list
197 of nodes in the catalog. The default, shown here, is to exclude any
198 files or directories that starts with a dot (.)
200 The only possible configuration parameter that you may need to change is
201 the one that maps a file to a data handler. This parameter is called
202 BES.catalog.TypeMatch, and is found in each of the data handler
203 configuration files, such as nc.conf. The default values should work.
205 The value of this parameter is a semicolon separated list that matches
206 the name you used in the BES.modules parameter with different datasets.
207 The BES uses regular expressions to identify different types of
208 datasets. In the example given in the file, any dataset name that ends
209 in '.nc' will be accessed using the netcdf hander (because the name 'nc'
210 is used here with the regular expression '.*\.nc'; note that the BES
211 uses regular expressions like Unix grep, not file globbing patterns like
212 a command shell). Since the name 'nc' was associated with the netcdf
213 module in the modules section, the netcdf module will be used to access
214 any dataset whose name that ends in '.nc'.
216 The regular expressions shown in the examples are simple. However, the
217 entire dataset name is used, so it's easy to associate different modules with
218 datasets based on much more than just the 'file name extension' even though
219 that is the most common case.
221 To test your regular expression for the TypeMatch parameter, or the
222 Include and Exclude parameters, use the supplied besregtest program.
223 Simply run besregtest to discover its usage.
225 ### Installing a custom handler/module
227 By default, The BES for Hyrax comes with a suite of handlers that read
228 a number of data formats. However, you can install custom handlers that
229 are not distributed by default. I'll use the SQL handler as an example.
231 Get the SQL handler source code from http://github.com/opendap/,
232 making sure that the version supports Hyrax. Expand the tar.gz file
233 and follow the instructions with the following caveat:
234 If you have installed the BES using a prefix other than /usr/local (the
235 default), make sure that the correct bes-config is being run. If you are
236 having problems compiling or linking the handler, try using not only
237 --prefix=... but also --with-bes=... when you configure the handler.
239 #### Install the newly-built handler.
241 Once built, install the handler using 'make install'. This will install
242 the BES module and a configuration file to use for that module. Each
243 module will have its own configuration file. In this case it is nc.conf
244 and installed in the _prefix_`/etc/bes/modules` directory. The next time
245 the BES is run, this configuration file will be read and the netcdf
246 module loaded. No modifications are necessary.
250 To test the server, open a new terminal window and start the bes by using
251 the bes control script besctl, which is installed in _prefix_`/bin`.
252 using the -c switch to name the configuration file. If the server standalone
253 starts correctly it should print something like the following to stdout:
256 [jimg@zoe sbin]$ besctl start
257 BES install directory: <prefix>/bin
258 Starting the BES daemon
259 PID: <process_id> UID: <uid of process>
262 Go back to your first window and run the bescmdln program. Use the -h (host)
263 and -p (port) switches to tell it how to connect to the BES.
266 [jimg@zoe bin]$ bescmdln -p 10002 -h localhost
268 Type 'exit' to exit the command line client and 'help' or '?' to display
274 Try some simple commands:
277 BESClient> show help;
288 Note that all commands must end with a semicolon except 'exit'.
290 Now try to get a DAP response using a handler you registered (the BES
291 supports a fairly complex syntax which I won't explain fully here):
294 BESClient> set container in catalog values n1, data/nc/fnoc1.nc;
295 BESClient> define d1 as n1;
296 BESClient> get das for d1;
299 String units "meter per second";
300 String long_name "Vector wind eastward component";
301 String missing_value "-32767";
302 String scale_factor "0.005";
305 String units "meter per second";
306 String long_name "Vector wind northward component";
307 String missing_value "-32767";
308 String scale_factor "0.005";
311 String units "degree North";
314 String units "degree East";
317 String units "hours from base_time";
320 String base_time "88- 10-00:00:00";
321 String title " FNOC UV wind components from 1988- 10 to 1988- 13.";
324 String Unlimited_Dimension "time_a";
331 If you got something similar (you would have used a different dataset name
332 from "data'nc'fnoc1.nc" and thus would get a different DAS response) the BES
333 is configured correctly and is running.
335 To stop the BES use the bes control script with the stop option:
338 [jimg@zoe sbin]$ besctl stop
341 Note: Constraints and the bes command line client
343 Constraints are added to the 'define' command using the modifier 'with' like
347 define d as nscat with nscat.constraint="WVC_LAT";
350 If there is a list of containers instead of just one, then there can be a
351 list of <container>.constraint="" clauses.
353 ## COPYRIGHT INFORMATION
355 The OPeNDAP BES code is copyrighted using the GNU Lesser GPL. See the
356 file COPYING or contact the Free Software Foundation, Inc., at 59 Temple
357 Place, Suite 330, Boston, MA 02111-1307 USA. Older versions of the BES were
358 copyrighted by the University Corporation for Atmospheric Research;
359 see the file COPYRIGHT_UCAR.