Esoteric Curio - BWPUG

Surge 2011 CFP Extension

nospam@example.com (Theo Schlossnagle) — Tue, 05 Apr 2011 22:41:11 -0400

The CFPs have been rolling in for Surge 2011; these are exciting times. It does, however, appear that our description of what we're looking for has produced a different set of submissions that what I expected. I think it might help to better understand what sessions were like last year and, luckily, we'll be releasing all of the Surge 2010 video footage this week. I apologize for the poor audio quality, we intend to pull in A/V recording professionals this year.

We've received some great proposals! However, a surprising amount of them are presentations about products. This conference is about problems and their solutions. It is a conference for practitioners by practitioners. Blood... in the mud. I want to (as we did last year) share our struggles for better collective experience.

Emphasis: Accepted proposals will demonstrate real-life scalability challenges and creative solutions. We love case studies and learning from our mistakes.

As such, with this context, I'm extending the Surge 2011 CFP deadline to April 17th.

What's your postgres search_path?

nospam@example.com (Theo Schlossnagle) — Sat, 09 Oct 2010 15:44:32 -0400

Hey you! PostgreSQL process running a query over there... Yeah you. What's your search path?

Hello? Why aren't you listening to me? Oh, just because you are busy running queries for someone else for hours means you don't have to take some time to answer my question? Apparently, that's a good enough excuse. You, yes you process ID 18883, need to respect my authority.

# echo '*postgres`namespace_search_path /s' | mdb -p 18883
0x9d5420:       noit_a29_n625680_noit, stratcon, public

I done told you.

Sorry. I didn't mean to get rough with you. You know I love you, right?

pg_amqp slides from PgEast2010

nospam@example.com (Theo Schlossnagle) — Tue, 30 Mar 2010 00:55:40 -0400

This past week I had the privilege of presenting along side many distinguished speakers at this year's PostgreSQL Conference East 2010 in Philadelphia, PA. I presented PostgreSQL: meet your queue which was received even more warmly than I had anticipated. I really think that cueing your database to publish over AMQP is the bees knees and it turns out I wasn't alone!

Asynchronous PostgreSQL Candy

nospam@example.com (Theo Schlossnagle) — Sun, 27 Dec 2009 13:49:32 -0500

I've put the first (intended) use of the pg_amqp setup to the test. So far I'm very pleased. While none of the code for the usage is open source the real "magic sauce" is open and has had a few bug fixes and bits of robustness added since I last posted. I can, however, describe the use and let your imagination run while.

The parts: PostgreSQL, RabbitMQ, and a bit of Java around Lucene. Our java program actually uses jetty to expose a tiny servlet that accepts search queries and returns json results making AJAX-style searches easy and fast.

The obvious question we all face in these sort of configurations is "how do keep our search indexes up-to-date?" Usually, we relax constraints (wisely so) and determine exactly "how up-to-date does out index needs to be." If you can answer this question with a number sufficiently far from zero, you've won. In our situation, the user experience must be able to search and find the updated items in database via search immediately after insert or update (within a 100ms or so).

Enter message queueing. The first thing we do is make our Java process connect to RabbitMQ, bind a queue and consume. The messages it consumes from the queue have instructions on the precise element that has changed which then causes an query against the database retrieving all the new data to be reindexed (in our case, it is much more than can be easily witnessed from a trigger on update) and updates the Lucene indexes.

The second updated we made is to PostgreSQL by installing pg_amqp. On the tables whose changes should induce reindexing, we add a (or augment the existing) trigger to call: amqp.publish('amq.direct', 'searchstuff', E'reindex\t' || NEW.rowid). Assuming we have a column called rowid, this will queue a message that looks like "reindex<tab>328432"

The neat part is that if the update is rolled back in the database, the message is never sent. Otherwise, we see the message from the our indexing and search app and viola.

I think this technique is really useful for people that are currently leveraging Lucene-based systems to provide powerful search functionality yet keep their data safe and secure in PostgreSQL.

Make the web a faster place. Pretty please.

nospam@example.com (Theo Schlossnagle) — Tue, 07 Jul 2009 10:36:59 -0400

Call to action! Make the web a faster place! Here's a short article on how I spent 45 minutes to improve user-perceived performance on our website. This is the low-hanging fruit of front-end web performance optimizations.

Most of you who read my blog are scalability or performance nuts. Most of you also cast the majority of your focus (like me) on the back-end infrastructure problems. Don't ignore the front-end when just a tiny bit of work can remove a huge amount of suck.

If everyone takes these steps, the web will be a more enjoyable place to visit.

Feb 2009 BWPUG Meeting: Stephen Frost, Column Level Permissions

nospam@example.com (Theo Schlossnagle) — Thu, 29 Jan 2009 12:42:15 -0500

Robert Treat was kind enough to coordinate the next BWPUG meeting. Here are his words, be there or be square!

UPDATED: the date of the meeting is 2009-02-09.

Welcome back everyone!

The last few months have been a busy time for everyone, but it's time to get focused again and get meeting again. To that end, for the BWPUG February meeting, we have lined up Stephen Frost to give a talk on Column Level Permissions in Postgres. Column level permissions are a new feature that Stephen authored which was recently committed into Postgres and will be available in the upcoming Postgres 8.4 release.

Stephen Frost is a Principal with Noblis, Inc., a non-profit organization working in the public interest supporting local, state, and federal Government. His work includes system architecture, system design, programming, unix administration, database administration, and management of the Noblis Innovation and Collaboration Center Computing Resources. As a PostgreSQL contributor, Stephen implemented roles support in 8.1 to replace the existing user/group system, and SQL column-level privileges in 8.4. As a PostGIS contributor and committer, Stephen updated the TIGER Geocoder for TIGER/Line and will be introducing a new version which works with the TIGER/Shapefile data at PGCon 2009.

Oh, and as a reminder, meetings are held at OmniTI world headquarters in Columbia, Maryland (http://omniti.com/is/here for details). We'll look to start around 6:30PM, and we'll have pizza and drinks available. We look forward to seeing everyone again.

BWPUG: The essential PostgreSQL.conf

nospam@example.com (Theo Schlossnagle) — Fri, 08 Aug 2008 13:55:44 -0400

Howdy folks,

This is a reminder that our monthly meetup is scheduled to take place this coming MONDAY, August 11th. As requested, we've moved the meetings from Wednesday to Monday to facilitate some of the would-be-attendees that have contacted me out of band.

This month's presentation is titled "The essential PostgreSQL.conf". With almost 200 configuration parameters, some people might think the postgresql.conf is a bit heady, but the truth is there are only about 2 dozen that you really need for everyday use. This talk will discuss the different types of configuration settings, and give an overview of the ones you'll want to know when running PostgreSQL. Speakers for the talk are Greg Smith, Software Engineer at Truviso, and Robert Treat, Database Architect at OmniTI.

Look forward to seeing you all there!

2008-08-11 @ 6:30pm
OmniTI
7070 Samuel Morse Dr. Ste 150
Columbia, MD 21046

Best regards,

Theo

OSCON2008 Presentation

nospam@example.com (Theo Schlossnagle) — Thu, 24 Jul 2008 03:58:09 -0400

Hello from OSCON. I gave my full-stack introspection crash course talk today. It has been quite a while since I've presented anything in a 40 minute format, but I think the talk went quite well. I got a lot of positive feedback.

I decided to take a risky approach inspired by dtrace.conf(08) by demonstrating dtrace on a live, mission-critical system we run at OmniTI. The risks of this are: network connections flake out, dtrace doesn't work correctly or I do something stupid and cause some service unavailability. Well, as I use dtrace just about every day, I wasn't worried about the breaking things. And while my network connection winked out for about one minute and dtrace has some annoying sub-second aborts, I think the demonstration was quite effective.

Many people gave positive commentary at the end and afterward. People asked for the slide to be put online... and while they have no real content of value (as the demo was everything), I put them here anyway:

OSCON2008 Full-stack Introspection Crash Course

view presentation (tags: oscon dtrace)

In addtion to the slide stack, I've included the simple scripts that I used during the demonstration. I ran qps.d, query_speed.d, and query_speed2.d on the database server and r3.sh, perl.d, pcpu.d, papcpu.d, and papcpu2.d on the web server. These require The Devel::DTrace perl module, PostgreSQL patches, and Apache 2.2.8 patches. Some of them are approximations of correctness, so weigh the output appropriately (the perl ones). Enjoy!

BWPUG Meetup Reminder

nospam@example.com (Theo Schlossnagle) — Tue, 13 May 2008 20:49:56 -0400

Hi all!

Just a friendly reminder that we'll be having our first meetup tomorrow as planned. I thought as a good kick-off we could all collaboratively share what we do with PostgreSQL. We'll start off with a whirlwind tour of how OmniTI uses PotsgreSQL, taking a brief look at ZFS, DTrace and large datasets. After that I think it would be good to get to know each other -- maybe we'll hit a local pub afterwards!

I look forward to seeing you there!

Meetup starts at 6:30pm
7070 Samuel Morse Dr. Ste 150
Columbia, MD 21046

If you have issues getting in the building, ring me on my cell -- it will be posted on the doors.

Best regards,

Theo

Starting the Baltimore/Washington PostgreSQL User Group

nospam@example.com (Theo Schlossnagle) — Mon, 14 Apr 2008 13:49:42 -0400

On the second Wednesday of every month, the Baltimore/Washington PostgreSQL User Group will meet at 7070 Samuel Morse Drive, Ste 150 in Columbia, Maryland. Meetings start at 6:30pm and go until around 8:30pm. I am pretty excited about this and pleased to offer up OmniTI's facilities for this. I'm excited about the opportunity to share what I've learned, educate and grow the PostgreSQL community and learn from others in it. This is going to be "good stuff."

Our first meeting will be held on May 14th, 2008. Mark your calendars. Also, subscribe to the mailing list at: bwpug@postgresql.org.