|
|
|
Posted
5 days
ago
Do you use PostgreSQL and truly believe it’s “the world’s most advanced open source database” and that its upcoming 9.3 release will make it even more awesome? Do you also use Python and believe it’s “an
... [More]
easy to learn, powerful programming language” with “elegant syntax” that makes it an ideal language for developing applications and tools around PostgreSQL, such as Pyrseas?
Then we could use your help. For starters, we want to add support for the MATERIALIZED VIEWs and EVENT TRIGGERs coming up in PG 9.3.
We have also been requested to add the capability to load and maintain “static data” (relatively small, unchanging tables) as part of yamltodb, so that it can be integrated more easily into database version control workflows.
And for the next release, Pyrseas 0.7, we’d like to include the first version of the database augmentation tool which will support declarative implementation of business logic in the database–starting off with audit trail columns. Some work has been done on this already, but it needs integration with the current code and tests.
Or perhaps coding is not your forte, but you’re really good at explaining and documenting technical “stuff”. Then you could give us a hand with revamping the docs, maybe writing a tutorial so that users have a smooth ride using our tools.
Or maybe you have your own ideas as to how improve the PostgreSQL version control experience. We’d love to hear those too.
If you’d like to help, you can fork the code on GitHub, join the mailing list and introduce yourself, or leave a comment below.
Filed under: PostgreSQL, Python, Version control [Less]
|
|
Posted
6 days
ago
Backup and recovery in Postgres-XC has some parallels to PostgreSQL, but with its own wrinkles.
read more
|
|
Posted
6 days
ago
Over the last decade, Greenplum, Vertica, Everest, Paraccel, and a number of non-public projects all forked off of PostgreSQL. In each case, one of the major changes to the forks was to radically change data storage structures in order to
... [More]
enable new functionality or much better performance on large data. In general, once a Postgres fork goes through the storage change, they stop contributing back to the main project because their codebase is then different enough to make merging very difficult.
Considering the amount of venture capital money poured into these forks, that's a big loss of feature contributions from the community. Especially when the startup in question gets bought out by a company who buries it or loots it for IP and then kills the product.
More importantly, we have a number of people who would like to do something interesting and substantially different with PostgreSQL storage, and will likely be forced to fork PostgreSQL to get their ideas to work. Index-organized tables, fractal trees, JSON trees, EAV-optimized storage, non-MVCC tables, column stores, hash-distributed tables and graphs all require changes to storage which can't currently be fit into the model of index classes and blobs we offer for extensibility of data storage. Transactional RAM and Persistent RAM in the future may urge other incompatible storage changes.
As a community, we want to capture these innovations and make them part of mainstream Postgres, and their users part of the PostgreSQL community. The only way to do this is to have some form of pluggable storage, just like we have pluggable function languages and pluggable index types.
The direct way to do this would be to refactor our code to replace all direct manipulation of storage and data pages with a well-defined API. This would be extremely difficult, and would produce large performance issues in the first few versions. It would, however, also have the advantage of allowing us to completely solve the binary upgrade of page format changes issue.
A second approach would be to do a MySQL, and build up Foreign Data Wrappers (FDWs) to the point where they could perform and behave like local tables. This may be the more feasible route because the work could be done incrementally, and FDWs are already a well-defined API. However, having Postgres run administration and maintenance of foreign tables would be a big step and is conceptually difficult to imagine.
Either way, this is a problem we need to solve long-term in order to continue expanding the places people can use PostgreSQL. [Less]
|
|
Posted
6 days
ago
If anyone doubts the total commitment of the Postgres project to quality and correctness, let them be reassured by this completely correct but decidedly pedantic commit. I dips me lid to Tom Lane and Thom Brown.
|
|
Posted
6 days
ago
New feature introduced in PostgreSQL 9.3Beta 1 i.e. "Disk page checksums". Thanks to authors Simon Riggs, Jeff Davis & Greg Smith. In earlier releases, if there's any data corruption block on disk it was silently ignored until any
... [More]
pointer arrives on it or some wrong results shown by the queries. Now, data corruption detected beforehand by triggering WARNING message instead of silently using or waiting for hit on the corrupted block.
Disk page checksums feature implementation is unique, its not plug-able like EXTENSIONs its selectable feature. That's if you need your database should be under monitory umbrella of data corruption then it should be enabled at the time of cluster initialization not on existing or running cluster. Below's the example how it works.
Initialize the cluster with checksums:
initdb [OPTION]... [DATADIR]
........
-k, --data-checksums use data page checksums
initdb -D data_directory -k
Now, any data corruption found will be notified as below:
postgres=# select * from corruption_test;
WARNING: page verification failed, calculated checksum 63023 but expected 48009
ERROR: invalid page in block 0 of relation base/12896/16409
In earlier version,just an error message.
postgres=# select * from corruption_test where id=1;
ERROR: invalid page header in block 0 of relation base/12870/18192
That's cool right....
So, how do you know whether disk page checksums enabled on the cluster or not ?
As of now, there's no pg_catalog to store such information or any files created in the $PGDATA directory, only pg_control file will hold that information. Using pg_controldata utility you can know about it.
$ export PGDATA=/usr/local/pg93beta/data
$ pg_controldata
....
....
....
Data page checksum version: 1
Some points on Disk page checksums:
1. Temp tables are excluded from checksums checks.
2. There's performance overhead if checksums enabled as per the PG documentation.
3. Once enabled checksums on a cluster cannot be rolled back.
Thanks
Raghav [Less]
|
|
Posted
6 days
ago
The Call for Papers for DjangoCon US 2013 is now open.
|
|
Posted
7 days
ago
Word of warning: this blogpost is about thing related to Bash (well, maybe other shells too, didn't really test), but since I found it while doing Pg work, and it might bite someone else doing Pg related work, I decided to add it to “postgresql" tag. So, due to some work I had to do, [...]
|
|
Posted
7 days
ago
The first pgCon Unconference is only 10 days away, and we now have room numbers. If you want to lead a topic ... or, more importantly, if you want someone else to lead a topic ... please add your topic ideas to the wiki page! We're
... [More]
expecting over 100 people at the unconference, given that pgCon registration is up above 225, more than 10% higher than last year at this point.
Second, my article on the PostgreSQL 9.3 Beta is up at LWN.net (subscription required, or wait 2 weeks). [Less]
|
|
Posted
7 days
ago
A year ago I wrote article about PostgreSQL history. Original article is in Czech language, but there is a link to Google translated article
|
|
Posted
7 days
ago
After playing a bit with the Redis FDW and the Redis command extension, Josh Berkus was slightly dissatisfied. He was looking for a way to map the possibly huge set of values in a single Redis object to a set of rows on a PostgresSQL table, and the
... [More]
Redis FDW currently maps each object to a single row - as an array if it's appropriate. Of course, we could call unnest () on the array, but it seems roundabout to construct an array only to have to unpack it immediately. These aren't terribly cheap operations.
I've been thinking a bit about his complaint, and I think this will be doable. What we'll need is a new table option that specifies the key and designates the table as one sourced from a single key rather than a range of keys. Say we call this option singleton_key. Then we might do something like:
CREATE FOREIGN TABLE hugeset (value text)
SERVER localredis
OPTIONS (tabletype 'list', singleton_key 'myhugelist');
This option would be incompatible with the tablekeyset and tablekeyprefix options. If given, the key won't be looked up at all. We would simply use the given key and return the corresponding list of values. That would make selecting from such a table faster - possibly lots faster. For scalars, sets and lists, the table would have one column. In the case of a scalar there would only be one row. For zsets, it would have two columns - one for the value and one for the score. Finally, for hashes it would have two, one for the property and one for the value.
This looks like a useful possible enhancement. [Less]
|
Copyright
©
2013
Black Duck Software, Inc.
and its contributors, Some Rights Reserved. Unless otherwise marked, this work is licensed under a
Creative Commons Attribution 3.0 Unported License
. Ohloh
®
and the Ohloh logo are trademarks of
Black Duck Software, Inc.
in the United States and/or other jurisdictions. All other trademarks are the property of their respective holders.