Posted
about 20 hours
ago
Update
Some have commented that in using nginx to do your load balancing you lose session affinity since nginx won't send it's users to the same backend. This can hurt performance since each zeo client would potentially have to cache
... [More]
a single user's specific objects.
If you find this to be a problem, there is a sticky nginx module that will handle this for load balancing. With this, each browser will be sent to the same backend.
Introduction
Why muck around with HAProxy and Varnish when you can have nginx do it all for you. The setup is easy and it's a lot easier to maintain.
Installing nginx with buildout
You can setup nginx fairly easily with buildout. The only fancy part of our setup is that we're going to include the nginx cache purge module.
Add the cache purge part to your buildout
[ngx_cache_purge]
recipe = hexagonit.recipe.download
url = http://labs.frickle.com/files/ngx_cache_purge-1.1.tar.gz
strip-top-level-dir = true
I needed the pcre source to compile also
[pcre-source]
recipe = hexagonit.recipe.download
url = ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.00.tar.gz
strip-top-level-dir = true
and the nginx part
[nginx-build]
recipe = hexagonit.recipe.cmmi
url = http://nginx.org/download/nginx-0.8.45.tar.gz
configure-options =
--with-http_stub_status_module
--conf-path=${buildout:directory}/settings/nginx.conf
--error-log-path=${buildout:directory}/var/log/nginx-error.log
--pid-path=${buildout:directory}/var/nginx.pid
--lock-path=${buildout:directory}/var/nginx.lock
--with-pcre=${pcre-source:location}
--with-http_ssl_module
--add-module=${ngx_cache_purge:location}
and finally, add it all to the parts directive
parts =
...
pcre-source
ngx_cache_purge
nginx-build
...
After you re-run buildout, you'll be able to run nginx by issuing a command like this:
./parts/nginx/sbin/nginx -c /path/to/configuration/nginx.conf
nginx configuration
Here is a simple sample configuration for nginx configured with load balancing and caching. It can obvious get as complicated as you want it, but I definitely think this is easier than managing haproxy, varnish and nginx.
pid /path/to/buildout/var/nginx.pid;
lock_file /path/to/buildout/var/nginx.lock;
worker_processes 2;
daemon off;
events {
worker_connections 1024;
}
error_log /path/to/buildout/var/log/nginx-error.log warn;
# HTTP server
http {
server_names_hash_bucket_size 64;
# this is how you do simple round robin load balancing with nginx.
# you can define as many backup servers as you'd like here.
upstream plone {
server 127.0.0.1:8080;
server 127.0.0.1:8081;
}
access_log /path/to/buildout/var/log/main-access.log;
# can specify multiple cache paths for different resources/paths/proxies
# if needed..
# the levels=1:2 just means it'll store the cache'd files 2 levels down in
# the folder structure
proxy_cache_path /var/www/cache levels=1:2 keys_zone=thecache:100m max_size=1000m inactive=600m;
proxy_temp_path /var/www/cache/tmp;
# Here is the caching purge handling. Purge request come in here
server {
listen 8089;
server_name www.example.com;
access_log /path/to/buildout/var/log/purge.log;
location / {
allow 127.0.0.1;
deny all;
proxy_cache_purge thecache $scheme$proxy_host$request_uri;
}
}
server {
listen 80;
server_name www.example.com;
# log for cache hits.
log_format cache '***$time_local '
'$upstream_cache_status '
'Cache-Control: $upstream_http_cache_control '
'Expires: $upstream_http_expires '
'"$request" ($status) '
'"$http_user_agent" ';
access_log /path/to/buildout/var/log/cache.log cache;
# Enable gzip compression
gzip on;
gzip_min_length 1000;
gzip_proxied any;
gzip_types text/xml text/plain application/xml;
# Show status information on /_main-status
location = /_main_status_ {
stub_status on;
allow 127.0.0.1;
deny all;
}
# do not cache when users are logged in..
proxy_cache_bypass $cookie___ac;
location / {
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 0;
client_body_buffer_size 128k;
proxy_send_timeout 120;
proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
proxy_connect_timeout 75;
proxy_read_timeout 205;
http://plone/VirtualHostBase/http/www.example.com:80/Plone /VirtualHostRoot/;
proxy_cache_bypass $cookie___ac;
proxy_cache thecache;
proxy_cache_key $scheme$proxy_host$request_uri;
}
}
} [Less]
Posted
1 day
ago
This little tutorial explains how to increase size of existing .vdi virtual hard drive file once the guest OS is already installed on it. I’ve taken a KISS approach describing each end every single step.
Posted
1 day
ago
This is the last post in my software releases series (at
least, the last as I originally planned the series).
In the last 5 articles, you've seen a couple of places where there's some
irritating boilerplate. What's supposed to go
... [More]
into your setup.py? How
did you start off with a buildout.cfg again? Oh, don't forget the
CHANGES.txt and a basic README.txt. The solution for that is
something I normally call a skeleton. I mentioned skeletons earlier in a
Dutch python user group talk on practical project automation.
A skeleton is a basic project structure with a README, a couple of
directories, some python files to start off with and whatever else you want to
put in there. You give it some parameters (project name, website address,
whatever) and it'll fill in the blanks with those parameters.
Django has a manage.py startapp, but there's a generic python tool called
PasteScript. PasteScript's
documentation almost exclusively talks about running wsgi applications, but it
actually can create project skeletons for you. The best way to get started is
to look at ZopeSkel, which has a
large collection of skeletons. Download it and see what it can do.
There are two core advantages to using skeletons either for yourself or
within your community or within your company:
Boilerplate reduction. Basic files get created and set up for you.
Laziness works to your advantage. "Just start the project from a skeleton"
gives you something reasonably solid and neat. The alternative is to just
start off with a python file and a sub directory and "remember" to add the readme
and so later.
Best practice. Pour your knowledge into a skeleton. Figure out the way
you want to set up your projects once and reap the fruits many times over.
You're probably copy-pasting apache config files from one project to the
other: why not keep the latest and greatest config file in the skeleton?
For me, this is the number one advantage: you've got a place to collect
your project setup best practice. And having that place to put it means
that more best practice gets attracted!
Want to start your own skeleton? What I'd recommend is to start with a
ZopeSkel download and to look at
their code and how to set it all up. Then start your own. I worked for Zest
software and started "zestskel" there. I worked
for The Health Agency afterwards and
started "thaskel" there. Now I work at Nelen en Schuurmans so I've started "nensskel" here :-)
Some examples of what I get when I create a new django site with
nensskel (after giving it a project name):
Basic setup.py with project name filled in. Long description is read
from the readme and changelog. Some common dependencies are pre-filled.
buildout.cfg which sets up django, creates apache config files, contains
package-versioning best practice setup, contains some commonly used tools,
etc.
Readme, changelog, todo file. Of course with the project name in there.
Basic apache config file. In here there's also the configuration that's
needed for django's static files. And some caching setup. And the wsgi
configuration. And the setup needed for django-compressor.
A directory with the actual django app (empty models.py, views.py
and so). A bit like how django's manage.py startapp does it, but now
with our own defaults.
Our own defaults? Yes, for instance the boilerplate needed in
settings.py for our django-staticfiles css/js setup.
You definitively don't want to type that by hand.
Pre-created PROJECTNAME/templates/PROJECTNAME,
PROJECTNAME/media/PROJECTNAME and PROJECTNAME/fixtures/ directories.
Ready-made test setup just the way we like it.
So: make it easy to do the right thing. Let laziness work in your
favour. Start a skeleton today! [Less]
Posted
2 days
ago
Easier configuration and Python 2.7 support
Posted
3 days
ago
I JUST heard about Adobe’s acquisition of Day Software and have to admit my first reaction was total disappointment. I always admired Day’s commitment to architecture and standards. Day is one of the few upper upper tier web content management
... [More]
companies to stay focused on the web — not just as a place [...]
Related posts:New ECM Interoperability Standard Proposal on AIIM There is a new proposal for a standards based...
Whatever happened to the URL? Even back when I was developing websites in 1998,...
iECM: Interoperable Enterprise Content Management iECM is a new standard being developed through AIIM...
Related posts brought to you by Yet Another Related Posts Plugin. [Less]
CMF 
i18ndude 
Ingeniweb Products Kupu Zope Object Database (ZODB) Five Plone4Artists OpenPlans Grok
