Posted
5 days
ago
by
nor...@blogger.com (Dave Page)
FOSDEM is a major Free and Open Source event held annually in Brussels, Belgium, and attended by around 4000 people. As in recent years, the PostgreSQL project will have a devroom where we will be presenting a number of talks. The event will be held
... [More]
on the 6 - 7th February 2010.
We're looking for developers, users and contributors to submit talks for inclusion on the program. Any topic related to PostgreSQL is acceptable as long as it is non-commercial in nature. Suggested topics might include:
Migration of systems to PostgreSQL
Application development
Benchmarking and tuning
Spatial applications
Hacking the code
Data warehousing
New features
Tips and tricks
Replication
Case studies
We will have a number of 45 minutes slots, and may split one or more into 3 back-to-back 15 minute slots if we receive suitable proposals.
Please submit your proposals to:
fosdem@postgresql.eu
and include the following information:
Your name
The title of your talk (please be descriptive, as titles will be listed with ~250 from other projects)
A short abstract of one to two paragraphs
A short biography introducing yourself
Links to related websites/blogs etc.
The deadline for submissions is 22nd December 2009.
See you in Brussels! [Less]
Posted
6 days
ago
by
dp...@pgadmin.org
FOSDEM is a major Free and Open Source event held annually in Brussels, Belgium, and attended by around 4000 people. We're looking for PostgreSQL developers, users and contributors to submit talks for inclusion on the program.
Posted
6 days
ago
by
in...@sqlmaestro.com
SQL Maestro Group announces the release of PostgreSQL Data Wizard 9.12, a powerful Windows GUI solution for PostgreSQL data management.
Posted
6 days
ago
by
nor...@blogger.com (Bernd Helmle)
So i had just another PostgreSQL course at the LinuxHotel in Essen. As always, i enjoyed working there with people interested in learning PostgreSQL within a very nice atmosphere. The reason i'm blogging about this, is that i'm really impressed how
... [More]
PostgreSQL has gained momentum over the last three years.
Doing such courses a few times a year at the LinuxHotel and getting regular feedback from attendees, i noticed that PostgreSQL has reached various places in the german industry and administration. Its used in ERP, Inventory Control, Data Warehouses, Webapplications, Information Retrieval. Often PostgreSQL serves mission critical applications, sometimes its used to implement a satellite system along with some other proprietary product. In many installations this was the entry key for PostgreSQL at all, since this allowed people to get their experience with PostgreSQL in a production environment without a radical change. They used this opportunity to build up their knowledge and infrastructure they need to run PostgreSQL as a backend for their applications and to recognize its reliability and stability. There are examples out there, where PostgreSQL finally replaced the main system afterwards. [Less]
Posted
7 days
ago
At pgday there was this form you could fill to give speakers some feedback
about their talks. And that's a really nice way as a speaker to know what to
improve. And as Magnus was searching a nice looking chart facility in python
and I
... [More]
spoke about matplotlib, it felt like having to publish something.
Here is my try at some nice graphics. Well I'll let you decide how nice the
result is:
If you want to see the little python script I used, here it is: feedback.py,
with the data embedded and all...
Now, how to read it? Well, the darker the color the better the score. For
example I had 5 people score me 5 for Topic Importance on the Hi-Media talk
(in french) and only 3 people at this same score and topic for pg_staging
talk. The scores are from 1 to 5, 5 being the best.
The comitee accepted interesting enough topics and it seems I managed to
deliver acceptable content from there. Not very good content, because
reading the comments I missed some nice birds-eye pictures to help the
audience get into the subject. As I'm unable to draw (with or without a
mouse) I plan to fix this in latter talks by using ditaa, the DIagrams
Through Ascii Art tool. I already used it and together with Emacs
picture-mode it's very nice.
Oh yes the baseline of this post is that there will be later talks. I seem
to be liking those and the audience feedback this time is saying that it's
not too bad for them. See you soon :) [Less]
Posted
7 days
ago
by
nor...@blogger.com (Francisco Figueiredo Jr.)
Hi all!
I just installed an app which allows me to post direclty from my mobile.
Now I think I'll be able to write more frequently, or at least I hope so. :-)
Stay tuned!
- Posted using BlogPress from my iPhone
Posted
9 days
ago
So I had two bug reports about prefix in less than a week. It means several
things, one of them is that my code is getting used in the wild, which is
nice. The other side of the coin is that people do find bugs in there. This
one is about
... [More]
the behavior of the btree opclass of the type prefix range. We
cheat a lot there by simply having written one, because a range does not
have a strict ordering: is [1-3] before of after [2-4]? But when you know
you have no overlapping intervals in your prefix_range column, being able to
have it part of a primary key is damn useful.
Note: in 8.5 we should have a way to express contraint exclusion and have
PostgreSQL forbids overlapping entries for us. Not being there yet, you
could write a constraint trigger and use the GiST index to have nice speed
there, which is exactly what this constraint exclusion support is about.
It turns out the code change required is pretty simple:
- return (a->first == b->first) ? (a->last - b->last) : (a->first - b->first);
+ /*
+ * we are comparing e.g. '1' and '12' (the shorter contains the
+ * smaller), so let's pretend '12' < '1' as it contains less elements.
+ */
+ return (alen == mlen) ? 1 : -1;
This happens in the compare support function (see
Interfacing Extensions To Indexes) so that means you now have to rebuild
your prefix_range btree indexes, hence the version number bump. [Less]
Posted
9 days
ago
The Linux OOM Killer heuristic can be summed up as:
Run out of memory.
Kill PostgreSQL.
Look for processes that might be using too much memory, and kill them, hopefully freeing memory.
Notice that step #2 is independent of
... [More]
factors like:
Is PostgreSQL consuming a significant share of the memory resources?
Will killing PostgreSQL alleviate any memory pressure at all?
The reason for this is because linux, when under memory pressure, invokes the badness() function to determine which process to kill. One of the things that the function counts against a process is the total VM size of the process (no surprise), plus half of the total VM size of all of the children. That sounds reasonable — wait a minute, what about shared memory?
For every connection, PostgreSQL makes a child process, which shares memory with the parent; and if it is a big postgres instance, that can be a significant amount of memory. But the badness function counts the same shared memory against the parent 1+N/2 times, where N is the number of connections open! For example, if you have shared_buffers set to 1GB on an 8GB machine, and have 20 connections open, then the linux badness function thinks that PostgreSQL is using about 11GB of memory, when it is actually using 1GB!
It gets worse: killing a process does not free shared memory. And there are already administrator-controlled limits that start off fairly low (32MB, I think), so the administrator would have to make a mistake in order for shared memory to be the problem.
And it gets even worse: linux makes very little attempt to avoid getting into a bad situation in the first place. On most operating systems, if you ask for too much memory, malloc() will eventually fail, which will probably cause the application to terminate; or if the application is written to handle such conditions (like PostgreSQL), it will gracefully degrade. On linux, by default, brk() (the system call underlying malloc()) will happily return success in almost every situation, and the first hint that your application gets of a problem is a SIGKILL.
I have heard many excuses for this appalling set of behaviors, but none of them are satisfactory to me. Here is one explanation. Notice the implicit assumption is that the most important thing that a user may be running is a text editor, and it should just autosave every once in a while to avoid problems. Keeping processes actually running is apparently not important to Linux. The next general philosophy present in the email is that applications are stupid, and they will never get the error paths or rollback right, but PostgreSQL seems to do that just fine (it needs very little memory to roll back, and tries to free memory before that to make it less likely to run into a problem). And the author seems to think that the first hint of a system problem should be a SIGKILL based on some heuristic (which, in the case of linux, is seriously flawed, as I pointed out above).
Among other justifications are:
“You can configure linux to prevent this problem.” Sounds like MySQL, where you have to declare your tables to be transaction-safe. Why not safe by default, and configure for performance?
“Modern systems overcommit, and there are always edge cases where you get into problems.” OK, so keep them as edge cases, and at least attempt to avoid problem situations. If some bizarre set of circumstances, or maybe a malicious process, is able to cause memory problems on your system, then so be it. But well-behaved processes should get some indication of trouble prior to SIGKILL. And the real problem processes should be identified with some accuracy and reasonable intelligence (i.e. recognize that shared memory can’t be freed by the OOM Killer, and therefore shouldn’t be considered).
I have been complaining about this insanity for years:
http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg120596.html (2007)
http://lkml.org/lkml/2007/2/9/275 (2007)
http://lkml.org/lkml/2008/2/5/495 (2008)
It makes me happy that other free software operating systems are still under active development, as illustrated by the recent release of FreeBSD 8.0. I am not saying FreeBSD is better in every way than Linux, but I do believe that competition is important even in the free software world.
This post is obviously a rant, and an ill-informed one, at that. I am no expert on memory management, so let me know if I am way off-base. However, I have never heard reasonable justifications for these things in response to my emails to lkml, in personal discussions with kernel hackers or other knowledgeable Linux folk, or anywhere else. If you do have a reasonable explanation, I am more than willing to listen. [Less]
Posted
9 days
ago
by
fa...@zidsoft.com
CompareData 1.6.0 is released. New with this release is support for generating data sync script at folder level (for a group of tables)
Posted
10 days
ago
PostgreSQL RPM packages for Fedora-12 released:
http://yum.pgsqlrpms.org/news-fedora12-packages-released.php
