|
|
|
|
|
|
| Author |
Message |
Jack Diederich
*nix forums addict
Joined: 21 Feb 2005
Posts: 60
|
 Posted: Fri Feb 04, 2005 11:12 pm Post subject:
Re: Persistence design [was: RE: OT: why are LAMP sites slow?]
|
|
|
On Fri, Feb 04, 2005 at 09:09:46PM -0200, Carlos Ribeiro wrote:
| Quote: | On Fri, 4 Feb 2005 17:46:44 -0500, Jack Diederich <jackdied@jackdied.com> wrote:
On Fri, Feb 04, 2005 at 10:31:19AM -0800, Robert Brewer wrote:
Jack Diederich wrote:
If there is interest I'll follow up with some details on my own LAMP
software which does live reports on gigs of data and - you
guessed it -
I regret it is database backed. That story also involves why
I started
using Python (the prototype was in PHP).
I'd be interested, if only selfishly to hear more potential use cases
for *my* projects. ;)
It would be long so I'll spin it into a blog piece. They seem to be
all the rage these days *0.5 wink*
A blog piece of yours truly  (call it a blatant plug if you want)
http://pythonnotes.blogspot.com/2004/11/alternative-database-systems.html
snip, see there for a "see also" URL comment I dropped
I'm (slowly) designing a personal CMS that is intended to use
Subversion as its backend, for both blog & wiki style entries. It's
filesystem based, fast (the website can use the working copy for
nearly everything) and allows for versioning - something that its
difficult to implement properly with low-end RDBMSs. It looks
promising
|
If you are on linux you can mount SVN repositories as filesystems.
It only displays the most recent version of a file (and requires the
alpha version of Apache) but that would be a fine and simple way to
have Apache serve versioned things. Any time an article is edited or
added just write the file and Apache will serve it as static, blissfully
unaware the older versions exist. Later when you add functionality to
diff entries access the same repository with subversion tools.
-Jack |
|
| Back to top |
|
 |
Carlos Ribeiro
*nix forums beginner
Joined: 19 Feb 2005
Posts: 12
|
| Posted: Fri Feb 04, 2005 10:09 pm Post subject:
Re: Persistence design [was: RE: OT: why are LAMP sites slow?]
|
|
|
On Fri, 4 Feb 2005 17:46:44 -0500, Jack Diederich <jackdied@jackdied.com> wrote:
| Quote: | On Fri, Feb 04, 2005 at 10:31:19AM -0800, Robert Brewer wrote:
Jack Diederich wrote:
If there is interest I'll follow up with some details on my own LAMP
software which does live reports on gigs of data and - you
guessed it -
I regret it is database backed. That story also involves why
I started
using Python (the prototype was in PHP).
I'd be interested, if only selfishly to hear more potential use cases
for *my* projects. ;)
It would be long so I'll spin it into a blog piece. They seem to be
all the rage these days *0.5 wink*
|
A blog piece of yours truly (call it a blatant plug if you want)
http://pythonnotes.blogspot.com/2004/11/alternative-database-systems.html
"""
There was a time when a database meant a flat file, fixed record
repository. Indexes were added later, bringing better performance for
several tasks. During the sixties, hierarchical database systems were
developed, allowing to model complex real-life structures better. Even
today, old-style mainframe systems (such as IBM's IMS) are still in
production, managing huge databases. SQL was only invented in the
seventies, based on a mathematical formalization of high-level data
manipulation algorithms. Batch processing systems read and process
data in a sequential fashion, and normally do not need such
abstractions. But the new generation interactive systems really needed
them. And when PC-based client-server computing exploded in the 90's,
SQL kingdom was started.
"""
....
"""
In the middle of this, there is a unforeseen trend in the use of the
file system as a storage medium. Yes, the file system. Guess what?
Forget the FAT, please. Current file systems are much more stable and
efficient than older ones. Modern filesystems are hierarchical, and
can store arbitrary objects. Support for journaling, and better
metadata management means that the filesystem is now a better choice
for many situations. Several web publishing engines (blogs, wikis, and
even full-fledged content management systems) support filesystem-based
storage for text notes and documents, which were previously stored (in
a hackish and haphazardous way) into DB blobs. The full filename is
now a primary key, and flexible relationships between entities can be
expressed as hiperlinks.
"""
I'm (slowly) designing a personal CMS that is intended to use
Subversion as its backend, for both blog & wiki style entries. It's
filesystem based, fast (the website can use the working copy for
nearly everything) and allows for versioning - something that its
difficult to implement properly with low-end RDBMSs. It looks
promising :-)
--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@gmail.com
mail: carribeiro@yahoo.com |
|
| Back to top |
|
|
Jack Diederich
*nix forums addict
Joined: 21 Feb 2005
Posts: 60
|
| Posted: Fri Feb 04, 2005 9:46 pm Post subject:
Re: Persistence design [was: RE: OT: why are LAMP sites slow?]
|
|
|
On Fri, Feb 04, 2005 at 10:31:19AM -0800, Robert Brewer wrote:
| Quote: | Jack Diederich wrote:
*ding*ding*ding* The biggest mistake I've made most
frequently is using
a database in applications. YAGNI. Using a database at all has it's
own overhead. Using a database badly is deadly. Most sites would
benefit from ripping out the database and doing something simpler.
Refactoring a database on a live system is a giant pain in the ass,
simpler file-based approaches make incremental updates easier.
The Wikipedia example has been thrown around, I haven't looked at the
code either; except for search why would they need a database to
look up an individual WikiWord? Going to the database
requires reading
an index when pickle.load(open('words/W/WikiWord')) would
seem sufficient.
...
If there is interest I'll follow up with some details on my own LAMP
software which does live reports on gigs of data and - you
guessed it -
I regret it is database backed. That story also involves why
I started
using Python (the prototype was in PHP).
I'd be interested, if only selfishly to hear more potential use cases
for *my* projects. 
|
It would be long so I'll spin it into a blog piece. They seem to be
all the rage these days *0.5 wink*
| Quote: | One of my goals for Dejavu* (my ORM) is to abstract persistence to the
point that you can easily test your actual, live dataset against many
potential storage mechanisms (i.e. multiple DB's, but also shelve,
etc.). I need to finish the migration tools, but it's well on the way.
|
I just looked at the docs, looks like a nicer version of the homespun
one I use. If someone lends you the time machine set the year to 2001
and I'll use Dejavu instead. Reading the docs also makes me long for
class decorators (again). I use metaclasses to register tables to
databases, I see you inspect globals.
ex in a better world:
mydb = Database('foo')
@mydb.add_table
class Foo(Table):
col1 = int
col2 = str
@mydb.add_table
class Bar(Table):
foo_id = str
colb = unicode
*sigh* maybe in 2.5?
-Jack |
|
| Back to top |
|
|
Robert Brewer
*nix forums beginner
Joined: 21 Feb 2005
Posts: 49
|
| Posted: Fri Feb 04, 2005 5:31 pm Post subject:
Persistence design [was: RE: OT: why are LAMP sites slow?]
|
|
|
Jack Diederich wrote:
| Quote: | *ding*ding*ding* The biggest mistake I've made most
frequently is using
a database in applications. YAGNI. Using a database at all has it's
own overhead. Using a database badly is deadly. Most sites would
benefit from ripping out the database and doing something simpler.
Refactoring a database on a live system is a giant pain in the ass,
simpler file-based approaches make incremental updates easier.
The Wikipedia example has been thrown around, I haven't looked at the
code either; except for search why would they need a database to
look up an individual WikiWord? Going to the database
requires reading
an index when pickle.load(open('words/W/WikiWord')) would
seem sufficient.
....
If there is interest I'll follow up with some details on my own LAMP
software which does live reports on gigs of data and - you
guessed it -
I regret it is database backed. That story also involves why
I started
using Python (the prototype was in PHP).
|
I'd be interested, if only selfishly to hear more potential use cases
for *my* projects. ;)
One of my goals for Dejavu* (my ORM) is to abstract persistence to the
point that you can easily test your actual, live dataset against many
potential storage mechanisms (i.e. multiple DB's, but also shelve,
etc.). I need to finish the migration tools, but it's well on the way.
Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
* http://www.aminus.org/rbre/python |
|
| Back to top |
|
|
Google
|
|
| Back to top |
|
|
|
|
The time now is Thu Jan 08, 2009 9:56 pm | All times are GMT
|
|
Credit Cards | CreditCards | Watch Anime Online | Debt Consolidation | Debt Consolidation
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
|
|
Other DeniX Solutions sites:
Unix/Linux blog |
electronics forum |
medicine forum |
science forum |
|
|
Privacy Policy
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Forum index
»
Programming
»
python
