Tuesday, April 3, 2018

New Annotated Config Files for PostgreSQL 10


Teal Deer: The Annotated.conf has been updated for PostgreSQL 10, and it's a Github repo now.

13 years ago, for PostgreSQL 8.0, I released the first Annotated.conf because we had (at that time) over 140 configuration settings and most people had no idea how to set them.  Today Postgres has nearly twice as many (269), and the problem is twice as bad. More about that later.

The repository is intended to be a quick reference for how to figure out how to set a lot of these parameters.  It includes my tutorial slides in both Libreoffice and PDF format, and a giant table of settings and recommendations in either CSV format or as a PostgreSQL 10 database dump.  Accompanying each setting are my notes on why you'd change the setting and, if so, to what.

Probably more usefully, I've included two sample .conf files.  postgresql.10.simple.conf contains the 20 most commonly changed settings with detailed advice on how to set them.  extra.10.conf has the next 20 most-likely-to-be-changed settings, with somewhat more limited advice.  The idea is for you follow the instructions, and then drop one or both of these files into your include_dir configuration directory and have a more sensible configuration without messing with the preposterously long default configuration file that ships with most PostgreSQL packages.  Those 20 to 40 settings should cover 90% of the needs of 90% of users.

If you feel that my advice is inaccurate, incomplete, or out-of-date, then Github has both Issues and Pull Requests available for you to suggest replacements.

Now, I can't help but feel that the configuration situation with PostgreSQL is getting worse.  While the database has improved with some better sensible defaults, and some good auto-configuration code (for example, for transaction log size), overall both the number of settings and the number of settings most users have to care about keeps going up.

The worst part of this is settings added with absolutely no information on how to determine a reasonable level for the setting.  For example, these four data flushing settings were added in version 10:

#backend_flush_after
#bgwriter_flush_after
#checkpoint_flush_after
#wal_writer_flush_after


... but there is no concrete advice on how to set these. Yes, there are docs, but even as someone who has a solid grounding in database, memory, and filesystem performance, I'd hesitate to recommend any specific level for these for a specific problem.  I'd like to be optimistic and assume that an explanation is coming in Postgres 11 or so, but after four years there's still nothing on how to set vacuum_multixact_freeze_min_age.

In the meantime, hopefully annotated.conf gives you a good start on tinkering with your own PostgreSQL settings.



Monday, January 29, 2018

LWKD Has a New Home!

Since this isn't a particularly good place for it, I've moved Last Week in Kubernetes Development to its new home at lwkd.info.  That includes this week's edition of LWKD.

Part of the idea of the move is to be able to accept contributions to the publication.  Since it's on Github pages, I can accept them through the LWKD git repo.  I particularly could use some help setting up an RSS feed in some way that doesn't require restructuring the site around a static site generator.

Also, a guest writer for next week would be welcome; otherwise the next issue is likely to be light due to personal travel.

Monday, January 22, 2018

Last Week in Kubernetes Development: Week ending January 21

Community Meeting Summary

The demo for this week's meeting was Kubernetes running on Docker For Mac. The Docker.com folks have been hard at work enabling this, and the demo now looks pretty polished. Developers with Mac desktops should be able to easily use Kubernetes with their existing Docker for Mac workflows.

Jaice DuMars explained the delay of the alpha release and gave a 1.10 stats update (see below). Dan Williams updated folks on SIG-Network, who have been making a lot of changes, including adding IPv6 support. Phillip Wittrock updated everyone on what the Steering Committee is currently working on, especially creating template SIG charters so that all SIGs can create their own charters. If you have opinions about the organization and leadership of your SIG, please take the survey on SIG governance.

Kubernetes will be participating in Google Summer of Code with the CNCF this year. Please contact Ihor Dvoretskyi if you are interested in mentoring or know a student. SIG Intros and Deep Dives at KubeCon Europe will be announced soon. The project will have another "Meet Our Contributors" on February 7th, this one focused on helping out new contributors (contact Paris Pittman to participate).

The format for the Community meeting will also be changing slightly in the future. SIGs will be scheduled for updates per release cycle instead of ad hoc, and demo speakers will be asked to rehearse before the meeting.

Release Schedule

This was week 3 of version 1.10 development. This week should have included an early alpha release, mainly as a dry run for release packaging. However, it's been delayed because Branch Manager Caleb Miles had a painful bike accident and has been offline. An alpha release is expected this week.

Feature Freeze, which was supposed to be January 22nd, has also been delayed by one week because the Features Lead is still waiting for status clarification on some features from several SIGs. Final Feature Freeze deadline will now be on the 29th. Many SIGs have updated their features, though, and Ihor has created the Feature Tracking Board for version 1.10.

Feature Work

While 148 patches were merged last week, most of them were minor bug fixes (including at least ten for GCE support), cherry-picks for copying fixes across releases, typo fixes, and some doc and release note corrections. Among the interesting feature work was:

 

Deprecated

 

Version Updates

 

Other Merges

 

Graph of the Week

 

This week's graph, brought to you by Jorge Castro, is Approvers and the Approvers Histogram This graph shows you the number of pull request approvals in each repository, and the accompanying histogram shows you who did those approvals. Looks like @cblecker is our leading approver for last month.

Monday, January 15, 2018

Last Week in Kubernetes: Week ending January 14th

With several hundred active contributors, it's pretty hard to keep track of Kubernetes development.   It's hard for me, and I'm paid to keep track; I can't imagine that anyone else can do it, even if they contribute to Kubernetes.

What follows is an experimental publication.  I'm thinking of doing a development summary, every week or so, of what's happened in new features, deprecations, the community meeting, and more.  Tell me if this is useful to you.  If it is, I'll look at finding an official place to publish it where maybe other community members can contribute.

Last Week in Kubernetes: Week ending January 14, 2017

Community Meeting Summary


The community meeting was dominated by a discussion around whether all repos in the kubernetes namespace should be a part of the same automation, particularly merge automation.  Aaron Crickenberger (spiffxp) has been offering this to other repos in the Kubernetes namespace, but some teams, particularly Helm, are concerned about unexpected changes this might cause.  One goal of getting all repos on the same automation is to retire mungegithub.

Jacob Pavlik demonstrated the KQueen cluster manager.  Jaice DuMars went over release 1.10, which is in week 2 of 12 and will go into Feature Freeze on January 22nd, so get your features in!  And SIG Azure and SIG Node made reports.

Feature Work

Configurable Pod Process Namespace Sharing prepared for inclusion in 1.10 this week with the addition of a feature flag for PID namespace sharing. The --docker-disable-shared-pid was also removed from kubelet.

Kubelets can now be run in containers, allowing for a completely containerized Kubernetes install. Such installs are now passing e2e tests.

Support for raw block devices as persistent volumes moved ahead with the merge of iSCSI support for block volumes.

Deprecated

Docker 1.10 is no longer supported. The minimum docker version is now 1.11. While docker 1.10 was officially deprecated in release 1.9, the compatibility code has now actually been completely removed.

sig-cluster-lifecycle is gradually deprecating the /cluster directory in favor of having these cluster setup tools maintained outside of kubernetes/kubernetes. In 1.10, that will include removing the windows/, photon-controller/, libvirt-coreos/, and gce/container-linux/ subdirectories, with more to be removed in future releases.

Version Updates

 

Other Merges