|
|
|
|
|
|
| Author |
Message |
JF Mezei
*nix forums Guru
Joined: 21 Jul 2005
Posts: 2556
|
 Posted: Tue Jul 04, 2006 7:56 am Post subject:
XML vs indexed files
|
|
|
OK, some here, including at least one VMS engineer seem to be quite
enamoured by XML.
XML seems to be great for storing variable format/content records where
one or more fields may be missing from some records, and fields can be
all variable length.
However, how does one handle XML in a context where you want to be able
to search though XML "records" ?
For instance, I could store GPS waypoint in XML in a text file. But if I
want to extract all "records" that match certain criteria, how do I go
about it ? a massive brute force search ?
If I want to add new records as well as replace existing records with a
matching primary key, how would I go about it with XML ?
I can see XML is great for inter-platform exchanges of raw data. But
does it really have a use as a in-platform "database" ? |
|
| Back to top |
|
 |
Thierry Dussuet
*nix forums beginner
Joined: 11 May 2005
Posts: 29
|
| Posted: Tue Jul 04, 2006 9:49 am Post subject:
Re: XML vs indexed files
|
|
|
On 2006-07-04, JF Mezei <jfmezei.spamnot@teksavvy.com> wrote:
| Quote: | OK, some here, including at least one VMS engineer seem to be quite
enamoured by XML.
XML seems to be great for storing variable format/content records where
one or more fields may be missing from some records, and fields can be
all variable length.
However, how does one handle XML in a context where you want to be able
to search though XML "records" ?
For instance, I could store GPS waypoint in XML in a text file. But if I
want to extract all "records" that match certain criteria, how do I go
about it ? a massive brute force search ?
If I want to add new records as well as replace existing records with a
matching primary key, how would I go about it with XML ?
I can see XML is great for inter-platform exchanges of raw data. But
does it really have a use as a in-platform "database" ?
|
You might want to read a bit about XPath.
Here are some examples which use libxml, including an XPath example:
http://xmlsoft.org/examples/index.html
And an XPath introduction:
http://www.w3schools.com/xpath/default.asp
Thierry |
|
| Back to top |
|
|
davidc@montagar.com
*nix forums addict
Joined: 08 Jul 2005
Posts: 76
|
| Posted: Tue Jul 04, 2006 2:49 pm Post subject:
Re: XML vs indexed files
|
|
|
Thierry Dussuet wrote:
| Quote: | On 2006-07-04, JF Mezei <jfmezei.spamnot@teksavvy.com> wrote:
For instance, I could store GPS waypoint in XML in a text file. But if I
want to extract all "records" that match certain criteria, how do I go
about it ? a massive brute force search ?
If I want to add new records as well as replace existing records with a
matching primary key, how would I go about it with XML ?
I can see XML is great for inter-platform exchanges of raw data. But
does it really have a use as a in-platform "database" ?
You might want to read a bit about XPath.
|
XML has it's place. Despite XPath/SAX/DOM, you essentially still do a
brute force search for what you are looking for, and maybe worse, since
all the XML must be parsed and in-memory trees built. XML is
essentially a specially formatted (HTML-like, essentially) text file.
XML is great for data storage and exchange in an open and portable way,
dumping data extracts for web/AJAX applications, documents (such as
OpenOffice does), configuration files, etc.
But XML as a replacement for index files? No. Indexed files will
always give you better and faster retrieval of information, assuming
you're based on an index. XML does not have the concept of "primary
key". Adding or changing a record requires re-writing the entire XML
document. So indexed files will provide superior protection of
primary/unique keys, faster updates and faster record additions than
XML.
I like XML, but like any tool, it's not the best solution for all
applications. |
|
| Back to top |
|
|
cdf
*nix forums Guru
Joined: 25 Apr 2005
Posts: 303
|
| Posted: Tue Jul 04, 2006 3:36 pm Post subject:
Re: XML vs indexed files
|
|
|
JF Mezei wrote:
| Quote: | OK, some here, including at least one VMS engineer seem to be quite
enamoured by XML.
XML seems to be great for storing variable format/content records where
one or more fields may be missing from some records, and fields can be
all variable length.
However, how does one handle XML in a context where you want to be able
to search though XML "records" ?
For instance, I could store GPS waypoint in XML in a text file. But if I
want to extract all "records" that match certain criteria, how do I go
about it ? a massive brute force search ?
If I want to add new records as well as replace existing records with a
matching primary key, how would I go about it with XML ?
I can see XML is great for inter-platform exchanges of raw data. But
does it really have a use as a in-platform "database" ?
|
It is easy to write an XLST program, that takes an XML file as input,
extract the wanted information and format it as and HTML page. However,
this method is only suited for information that can easily be stored in
main storage. If the XML file is larger, then I would consider other
solutions.
Have you considered using a database? You do not have to spend $$$ if
you task is not that demanding. You could use free a DBMS like MySQL or
PostgreSQL. Alternatively small installations of Oracle and Microsoft
SQL are not that expensive any more.
What is your problem and how much data do you have? |
|
| Back to top |
|
|
Malcolm Dunnett
*nix forums addict
Joined: 09 Sep 2005
Posts: 95
|
| Posted: Tue Jul 04, 2006 4:56 pm Post subject:
Re: XML vs indexed files
|
|
|
In article <44aa8af7$0$67259$157c6196@dreader2.cybercity.dk>,
Karsten Nyblad <nospam@nospam.nospam> writes:
| Quote: |
Have you considered using a database? You do not have to spend $$$ if
you task is not that demanding. You could use free a DBMS like MySQL or
PostgreSQL. Alternatively small installations of Oracle and Microsoft
SQL are not that expensive any more.
Small ( up to 4GB database size ) Oracle databases are free, using |
Oracle 10g Express Edition:
http://www.oracle.com/technology/products/database/xe/index.html
Unfortunately it's only available for Windows and Linux, not VMS. |
|
| Back to top |
|
|
Randy Park
*nix forums beginner
Joined: 05 Jul 2006
Posts: 1
|
| Posted: Wed Jul 05, 2006 1:58 am Post subject:
Re: XML vs indexed files
|
|
|
On Tue, 04 Jul 2006 03:56:56 -0400, JF Mezei <jfmezei.spamnot@teksavvy.com> wrote:
| Quote: | OK, some here, including at least one VMS engineer seem to be quite
enamoured by XML.
XML seems to be great for storing variable format/content records where
one or more fields may be missing from some records, and fields can be
all variable length.
However, how does one handle XML in a context where you want to be able
to search though XML "records" ?
For instance, I could store GPS waypoint in XML in a text file. But if I
want to extract all "records" that match certain criteria, how do I go
about it ? a massive brute force search ?
If I want to add new records as well as replace existing records with a
matching primary key, how would I go about it with XML ?
I can see XML is great for inter-platform exchanges of raw data. But
does it really have a use as a in-platform "database" ?
|
There is a VMS oriented commerical product that will extract data from
an XML "text" file. It treats the XML file as just a file of sequential records.
You can then output the data in a formatted report, to an RMS file (yes, isam),
to an HTML web page, etc. If you have the appropriate networking and servers
set up the data can even be elsewhere on the network. It is a full blown data
extraction and report writing tool. Yes you can add records if the XML file is
on a VMS system (well you might have to do a new file and merge). Matching
primary keys is not a concept that applies to XML formatted data.
The name of the product is Xentis, sold by Graymatter Software. In the interest
of full disclosure I used to have a financial interest in the product, but no longer
do.
--
Posted via a free Usenet account from http://www.teranews.com |
|
| Back to top |
|
|
Barry
*nix forums Guru Wannabe
Joined: 10 Jul 2005
Posts: 149
|
| Posted: Wed Jul 19, 2006 5:28 pm Post subject:
XML on Expat
|
|
|
(I figured it's better to piggyback on this one than to start a new
topic)
I'm aware of the XML parser for VMS from Expat. I recently downloaded
the files and used the supplied descrip.mms file to build the OLB.
However, I haven't been able to figure out what else has to be done in
order to build an executable that will in fact parse an XML file. I
have 15 years experience with VMS (and MMS) so that's not a problem.
If anyone can tell me what's necessary to get a runnable program built
from the Expat sources I'll gladly create an MMS file to do it all.
Thanks! |
|
| Back to top |
|
|
Hoff Hoffman
*nix forums Guru
Joined: 22 Jul 2005
Posts: 572
|
| Posted: Wed Jul 19, 2006 7:13 pm Post subject:
Re: XML on Expat
|
|
|
Barry wrote:
| Quote: | (I figured it's better to piggyback on this one than to start a new
topic)
I'm aware of the XML parser for VMS from Expat. I recently downloaded
the files and used the supplied descrip.mms file to build the OLB.
However, I haven't been able to figure out what else has to be done in
order to build an executable that will in fact parse an XML file. I
have 15 years experience with VMS (and MMS) so that's not a problem.
If anyone can tell me what's necessary to get a runnable program built
from the Expat sources I'll gladly create an MMS file to do it all.
|
If you don't specifically need the expat XML parser, a port of the
libxml2 environebt is available at the Freeware V8.0 staging area, FWIW.
<http://www.hp.com/go/openvms/freeware/>
<http://www.xmlsoft.org/>
The Freeware port includes build procedures, etc.
I have finished a libxml2 2.6.24 port not to long ago, with yet more
pieces, and that'll be the newest version that ships on the Freeware
V8.0 distro.
--
And one question I ask myself when dealing with my own or the makefiles
of others (and whether based on DECset mms, mmk, or otherwise) centrally
involves the speed of a brute-force build on newer hardware as compared
with the effort involved in maintaining and debugging and using an
incremental build. Incremental builds are obviously beneficial and can
be really speedy, but brute-force builds on the hardware I regularly use
can be very fast, too, and are obviously rather less involved than the
makefiles, and the brute-force build can obviously entirely avoid some
of the "fun" of the more subtle bugs that I and others have occasionally
managed to introduce into our makefiles. There are always trade-offs,
of course. |
|
| Back to top |
|
|
Martin Vorlaender
*nix forums addict
Joined: 23 Jun 2005
Posts: 78
|
| Posted: Thu Jul 20, 2006 8:39 am Post subject:
Re: XML on Expat
|
|
|
Barry wrote:
| Quote: | I'm aware of the XML parser for VMS from Expat. I recently downloaded
the files and used the supplied descrip.mms file to build the OLB.
However, I haven't been able to figure out what else has to be done in
order to build an executable that will in fact parse an XML file.
|
Expat is not a complete XML parser, but a library that offers an API
to parse XML files. What you do with the API really depends on what you
want to do with the information extracted.
| Quote: | I have 15 years experience with VMS (and MMS) so that's not a problem.
If anyone can tell me what's necessary to get a runnable program built
from the Expat sources I'll gladly create an MMS file to do it all.
|
A simple program showing how to use the expat API is at
http://www.xml.com/1999/09/expat/src/line.c .
So, what do you want to do with the XML information?
cu,
Martin
P.S.: Sorry for not having answered your email. My private account went
offline just after receiving it.
--
One OS to rule them all | Martin Vorlaender | OpenVMS rules!
One OS to find them | work: mv@pdv-systeme.de
One OS to bring them all | http://www.pdv-systeme.de/users/martinv/
And in the Darkness bind them.| home: martin@radiogaga.harz.de |
|
| Back to top |
|
|
Barry
*nix forums Guru Wannabe
Joined: 10 Jul 2005
Posts: 149
|
| Posted: Fri Jul 21, 2006 11:27 am Post subject:
Re: XML on Expat
|
|
|
Thanks, Hoff. No, I don't specifically need Expat -- just that most of
the stuff I've seen raves about it. I'll try the pointers you've
specified.
Hoff Hoffman wrote:
| Quote: | Barry wrote:
(I figured it's better to piggyback on this one than to start a new
topic)
I'm aware of the XML parser for VMS from Expat. I recently downloaded
the files and used the supplied descrip.mms file to build the OLB.
However, I haven't been able to figure out what else has to be done in
order to build an executable that will in fact parse an XML file. I
have 15 years experience with VMS (and MMS) so that's not a problem.
If anyone can tell me what's necessary to get a runnable program built
from the Expat sources I'll gladly create an MMS file to do it all.
If you don't specifically need the expat XML parser, a port of the
libxml2 environebt is available at the Freeware V8.0 staging area, FWIW.
http://www.hp.com/go/openvms/freeware/
http://www.xmlsoft.org/
The Freeware port includes build procedures, etc.
I have finished a libxml2 2.6.24 port not to long ago, with yet more
pieces, and that'll be the newest version that ships on the Freeware
V8.0 distro.
--
And one question I ask myself when dealing with my own or the makefiles
of others (and whether based on DECset mms, mmk, or otherwise) centrally
involves the speed of a brute-force build on newer hardware as compared
with the effort involved in maintaining and debugging and using an
incremental build. Incremental builds are obviously beneficial and can
be really speedy, but brute-force builds on the hardware I regularly use
can be very fast, too, and are obviously rather less involved than the
makefiles, and the brute-force build can obviously entirely avoid some
of the "fun" of the more subtle bugs that I and others have occasionally
managed to introduce into our makefiles. There are always trade-offs,
of course. |
|
|
| Back to top |
|
|
Barry
*nix forums Guru Wannabe
Joined: 10 Jul 2005
Posts: 149
|
| Posted: Fri Jul 21, 2006 11:38 am Post subject:
Re: XML on Expat
|
|
|
Thanks for the pointer to the example, Martin. What I want to do with
the XML is eventually get parts of it (i.e., only certain data items)
imported into Excel for subsequent processing. I know that Excel can
import XML directly, but these files I'm dealing with are too big for
Excel (42MB), so I figured a VMS utility might be better able to let me
manipulate the data and create a smaller file that Excel can handle.
Martin Vorlaender wrote:
| Quote: | Barry wrote:
I'm aware of the XML parser for VMS from Expat. I recently downloaded
the files and used the supplied descrip.mms file to build the OLB.
However, I haven't been able to figure out what else has to be done in
order to build an executable that will in fact parse an XML file.
Expat is not a complete XML parser, but a library that offers an API
to parse XML files. What you do with the API really depends on what you
want to do with the information extracted.
I have 15 years experience with VMS (and MMS) so that's not a problem.
If anyone can tell me what's necessary to get a runnable program built
from the Expat sources I'll gladly create an MMS file to do it all.
A simple program showing how to use the expat API is at
http://www.xml.com/1999/09/expat/src/line.c .
So, what do you want to do with the XML information?
cu,
Martin
P.S.: Sorry for not having answered your email. My private account went
offline just after receiving it.
--
One OS to rule them all | Martin Vorlaender | OpenVMS rules!
One OS to find them | work: mv@pdv-systeme.de
One OS to bring them all | http://www.pdv-systeme.de/users/martinv/
And in the Darkness bind them.| home: martin@radiogaga.harz.de |
|
|
| Back to top |
|
|
Jean-François Piéronne
*nix forums beginner
Joined: 07 Jul 2005
Posts: 28
|
| Posted: Fri Jul 21, 2006 11:52 am Post subject:
Re: XML on Expat
|
|
|
Barry,
| Quote: | Thanks for the pointer to the example, Martin. What I want to do with
the XML is eventually get parts of it (i.e., only certain data items)
imported into Excel for subsequent processing. I know that Excel can
import XML directly, but these files I'm dealing with are too big for
Excel (42MB), so I figured a VMS utility might be better able to let me
manipulate the data and create a smaller file that Excel can handle.
|
Python on OpenVMS has interface to expat and libxml2, latest kits also
include pyExcelerator which allow to read/write Excel document.
IMHO, it is much easier (faster) to write this using Python than using C
code.
JF |
|
| Back to top |
|
|
Google
|
|
| Back to top |
|
|
|
|
The time now is Sun Nov 23, 2008 1:23 pm | All times are GMT
|
|
Loans | Debt Consolidation | Payday Loan | Loans | Personal Loans
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
|
|
Other DeniX Solutions sites:
Unix/Linux blog |
electronics forum |
medicine forum |
science forum |
|
|
Privacy Policy
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Forum index
»
Not Unix
»
VMS
