Posted
18 days
ago
First, the bad news. I didn't make as much progress over the weekend as I'd
hoped. In fact, I really achieved only one thing from that whole post and it was
a side-effect of some other developments.
I threaded the Canto interface
... [More]
not because I really wanted to take advantage of
multiple cores or get a speedup in the actually task of filtering and sorting
item. I threaded it to get the interface to be more responsive. To an extent, it
worked. The interface became more responsive, but only because in implementing
the thread model, I had to implement partial updates. It effectively removed the
problem with long lists by allowing you to use the interface when only part of
the total work was done.
I never really cared to ponder why, as a whole, performance didn't really work
out until I read David Beazley's slides on the
GIL (PDF). It really clicked with me why
I was still running into trouble, even with the threads.
Now, the good news. I converted the entirety of the threading model to use
processes, which avoids the GIL all together by communicating through
traditional pipes. No GIL locking required. Worst case scenario is a moderate
speed up (because the locking overhead is completely gone). Best case scenario
is a hefty speed up brought on by today's multi-core processors. I've noticed a
big difference on my Core2 even when one core is being dominated by another
process.
There are still some issues in the implementation (interested parties can glance
at the git log.
But as a whole, the process based code is not that much different than the
thread based code was. Everything is still protocolized but the protocol has to
be entirely comprised of pickle-able objects.
I don't anticipate too many problems directly related to the new process based
code, but it's possible. Also, you're going to take a hit in memory usage
because the same set of objects has to be synced between the two processes, but
the dumb process doesn't keep too much state by default so it's been minimized.
I also don't think it's enough to outweigh the memory gains made by keeping more
content on disk so overally the memory performance is worse than when threading,
but should still be better than 0.6.x.
New dependency for 2.5
To ease the use of processes, Python 2.5 users will have to install
python-processing. Fortunately, this module has been integrated into the Python
standard library in 2.6+ as the "multiprocessing" module.
Testing
As always, the code is in the experimental branch. If you 'git' it, feel free to
report on the mailing lists or #canto on irc.freenode.net.
Have fun, submit bugs! [Less]
Posted
22 days
ago
Since the last update, the merging of the validating config code, there have
been a number of improvements to the experimental source. Mostly bug fixes. A
lot of things that have been broken for awhile have been updated to use the
... [More]
new
data structures.
Sorts are fixed
Tag filters are fixed
tags should now be totally implicit
Sluggish reader scrolling fixed
More canto.extra sugar for adding content to the interface
The most work has been done on code documentation though. At this point,
basically every file has a large header explaining its purpose and highlighting
important blocks of code, in addition to the typical as-you-go interpretation
comments. I know I'm the only real developer on Canto at the moment, but I've
added hundreds of comments to explain design decisions and give a good overview
about the internal functioning of the project.
I'm excited to say that I think that the majority of the goals for 0.7.0 have
been reached and are working in the code. I hope to put a major dent in the
remaining areas this weekend, particularly in getting the new site /
configuration / styling documentation up to speed. It's going to be a full
rewrite to fix a lot of documentation deficiencies.
What remains
Minor amounts of code documentation (there are a few files left)
Website documentation rewrite
Style opt-out flag / option
Feed error reporting (will also keep Canto from trying to update 404'd or
otherwise unfetchable feeds every time it starts)
More complete config validation (i.e. validate filters/sorts that aren't
put into typical places)
Various canto.extra improvements for ease of use
Some of these are reflected in the bug
tracker.
I can also say that there are still some performance problems with atypical
usage. For example, if you open canto and immediately switch to another global
filter, it lags because it's trying to update from disk and refilter all of the
previously updated items. This can easily be resolved by setting your default
global filter correctly in the config so you don't have to immediately switch
and if you have to switch for some other reason (i.e. to find a read item when
your default filter is show_unread), the penalty is only a couple of seconds.
I am actively looking into making this work better ( have no fear =) ), but none
of the poor performance cases are significant enough to keep me from releasing.
Anyway, keep watching for a release some time in the nearish future, I'll
probably update here with a link to the experimental page with a beta before I
actually put out 0.7.0 for the distros to pick up.
In the meantime, have fun and submit bugs! [Less]
Posted
about 1 month
ago
I finally merged the validating configuration code into the experimental branch.
Experimental users might want to read the last few git-log entries after they
pull the changes but before they install the new code. In particular git
... [More]
log
a9c86. The content is mirrored in gitweb
here
The long and the short of it is that there's been some heavy changes to some
particulars of the config and if you're running your own filters and sorts or
are using sorts heavily then your config might need a little tweaking and the
official documentation hasn't been written up yet.
As I mention in the log, if you have trouble, feel free to mail the list or drop
by #canto and get some help straightening it out. [Less]
Posted
about 1 month
ago
I missed the last week's edition, so this has a little more scope, but
I've been pretty slammed at work so not much has been going on.
These past two weeks have brought a number of bugfixes in the experimental
branch, but the main
... [More]
improvements were the addition of the state change hook
(which will eventually be used for integration with other readers, like Google
Reader), and the start of the validating modular configuration code (in the
exp-cfg branch). I'm very pleased with how the new configuration code is going
and it should offer much more informative configuration error messages and is
much stricter so that strange crashes don't come from strange corner cases in
the config.
Unfortunately, I've gone down some developer blind alleys in the past week. The
first of which is that I finally got around to determining just how difficult
integrating with GReader is going to be. As a result, I'm going to have to delay
that until sometime after 0.7.0 because the update is already going to add
enough features with their corresponding bugs that I'd like to get it out there.
I am, however, going to add some changes that make GReader integration less
difficult before the release.
The second is getting Canto to work on FreeBSD. I'm a Linux user/developer by
choice and by trade, but I'm never averse to expanding Canto's range to similar
operating systems. With this, I struggled for a little while, and ended up
submitting a bug in the Python tracker about
it because the WINDOW taken from the Python object is somehow not valid.
I'm not sure if the Python tracker is really the best place for it, but I'm
hoping that someone who knows a lot more about Python internals will take a look
and help me out.
Any experienced FreeBSD programmers that want to take a look at it are certainly
welcome to take a look as well, my experience with FreeBSD is very limited. I
still hope that any changes won't be have to be very invasive so I can add
FreeBSD support without adding much complexity.
Have fun! Submit bugs! [Less]
Posted
about 1 month
ago
You can grab the 0.6.12 release in tar or OSX portfile format from the
download page.
Fix tag crash
Fix strange character weirdness
Fix/Improve HTML parser
Try UTF-8 for config before chardet
Another round of
... [More]
miscellaneous fixes. In particular a crash in switching tags,
some strange case in the drawing code where a valid multi-byte character has a
wcwidth() of -1, and it looks like either the semantics for using the built
in HTMLParser in Python have changed or it had a bug with CDATA content
that's now worked around.
In addition, provide UTF-8 as a sane default for reading the config, rather than
ASCII, before falling back on the chardet library, which should fix some
strangeness with tags coming up in wrong encodings.
Have fun! Submit bugs! [Less]
news aggregator feed unix xml feedreader atom rss python ncurses rdf extensible feedparser linux syndication reader opml
|
edit tags
