Comments on: Django vs feedparser on dates http://www.deadlybloodyserious.com/2007/09/feedparser-v-django/ Just another WordPress weblog Fri, 04 Jul 2008 14:56:44 +0000 http://wordpress.org/?v=2.5.1 By: Fazal Majid http://www.deadlybloodyserious.com/2007/09/feedparser-v-django/#comment-2261 Fazal Majid Sun, 16 Sep 2007 12:14:12 +0000 http://www.deadlybloodyserious.com/2007/09/django-vs-feedparser-on-dates/#comment-2261 Keep in mind the DST bug could well be in the feed generator. Quite a few will blithely assume UTC = Local Time +/- offset and not factor in DST. I have often seen feeds with timestamps 1 hour into the future, and added logic to my aggregator, Temboz, to automatically substract 1 hour if it detects this: http://www.temboz.com/temboz/fileview?f=temboz/normalize.py Keep in mind the DST bug could well be in the feed generator. Quite a few will blithely assume UTC = Local Time +/- offset and not factor in DST.

I have often seen feeds with timestamps 1 hour into the future, and added logic to my aggregator, Temboz, to automatically substract 1 hour if it detects this:

http://www.temboz.com/temboz/fileview?f=temboz/normalize.py

]]> By: Doug Hellmann http://www.deadlybloodyserious.com/2007/09/feedparser-v-django/#comment-2148 Doug Hellmann Mon, 03 Sep 2007 09:18:30 +0000 http://www.deadlybloodyserious.com/2007/09/django-vs-feedparser-on-dates/#comment-2148 If you're building an aggregator, you might want to have a look at <a href="http://www.doughellmann.com/projects/feedcache/" rel="nofollow">feedcache</a>. When combined with <a href="http://pypi.python.org/pypi/shove" rel="nofollow">shove</a>, it makes managing feeds for aggregators very easy. If you’re building an aggregator, you might want to have a look at feedcache. When combined with shove, it makes managing feeds for aggregators very easy.

]]> By: garth http://www.deadlybloodyserious.com/2007/09/feedparser-v-django/#comment-2140 garth Sun, 02 Sep 2007 23:55:50 +0000 http://www.deadlybloodyserious.com/2007/09/django-vs-feedparser-on-dates/#comment-2140 <code> Testing translation methods against 195 timestamps... _ludo: 0 _devex: 0 _kiopi: 0 _anders: 0 _gt: 0 _sam_and_zeb: 0 _garth: 4 </code> All the methods work, except the one I got from the <a href="http://code.djangoproject.com/browser/djangoproject.com/django_website/apps/aggregator/bin/update_feeds.py" rel="nofollow">Django community blog</a>. Oops. Now I'll have to collect raw timestamps so I can check dateutil versus feedburner...
Testing translation methods against 195 timestamps...
_ludo: 0
_devex: 0
_kiopi: 0
_anders: 0
_gt: 0
_sam_and_zeb: 0
_garth: 4

All the methods work, except the one I got from the Django community blog. Oops.

Now I’ll have to collect raw timestamps so I can check dateutil versus feedburner…

]]> By: mikeal http://www.deadlybloodyserious.com/2007/09/feedparser-v-django/#comment-2136 mikeal Sun, 02 Sep 2007 21:38:25 +0000 http://www.deadlybloodyserious.com/2007/09/django-vs-feedparser-on-dates/#comment-2136 I think i ended up scratching the use of the feedparser _parsed attributes all together. The parsing code there just isn't as robust and dateutil.parser.parse(). For what it's worth I've never had problems with dateutil's parsing code, especially for ISO8601 parsing which is usually what feeds use. I think i ended up scratching the use of the feedparser _parsed attributes all together. The parsing code there just isn’t as robust and dateutil.parser.parse().

For what it’s worth I’ve never had problems with dateutil’s parsing code, especially for ISO8601 parsing which is usually what feeds use.

]]> By: Sam Ruby http://www.deadlybloodyserious.com/2007/09/feedparser-v-django/#comment-2133 Sam Ruby Sun, 02 Sep 2007 15:26:48 +0000 http://www.deadlybloodyserious.com/2007/09/django-vs-feedparser-on-dates/#comment-2133 <a href="http://intertwingly.net/blog/2007/09/02/Dealing-With-Dates" rel="nofollow">datetime.datetime.utcfromtimestamp(calendar.timegm(feedparser_timestamp))</a> datetime.datetime.utcfromtimestamp(calendar.timegm(feedparser_timestamp))

]]> By: Sam Ruby http://www.deadlybloodyserious.com/2007/09/feedparser-v-django/#comment-2132 Sam Ruby Sun, 02 Sep 2007 15:22:41 +0000 http://www.deadlybloodyserious.com/2007/09/django-vs-feedparser-on-dates/#comment-2132 <strong>Dealing With Dates...</strong> Simon Willison: Django vs feedparser on dates. Some useful tips in the comments. I find Python’s timezone stuff endlessly frustrating: I know it can do what I want, but it always take... Dealing With Dates…

Simon Willison: Django vs feedparser on dates. Some useful tips in the comments. I find Python’s timezone stuff endlessly frustrating: I know it can do what I want, but it always take…

]]> By: Zebziggle http://www.deadlybloodyserious.com/2007/09/feedparser-v-django/#comment-2131 Zebziggle Sun, 02 Sep 2007 13:04:36 +0000 http://www.deadlybloodyserious.com/2007/09/django-vs-feedparser-on-dates/#comment-2131 Try again with formatting ... <code> if hasattr(content, 'modified') and \ content.modified != None: feed.lastModified = \ datetime.datetime.utcfromtimestamp( calendar.timegm(content.modified)) </code> Try again with formatting …


if hasattr(content, 'modified') and \
content.modified != None:
feed.lastModified = \
datetime.datetime.utcfromtimestamp(
calendar.timegm(content.modified))

]]> By: Zebziggle http://www.deadlybloodyserious.com/2007/09/feedparser-v-django/#comment-2130 Zebziggle Sun, 02 Sep 2007 13:02:59 +0000 http://www.deadlybloodyserious.com/2007/09/django-vs-feedparser-on-dates/#comment-2130 Mine looks like this: <code> if hasattr(content, 'modified') and content.modified != None: feed.lastModified = datetime.datetime.utcfromtimestamp(calendar.timegm(content.modified)) </code> Mine looks like this:


if hasattr(content, 'modified') and content.modified != None:
feed.lastModified = datetime.datetime.utcfromtimestamp(calendar.timegm(content.modified))

]]> By: ludo http://www.deadlybloodyserious.com/2007/09/feedparser-v-django/#comment-2128 ludo Sun, 02 Sep 2007 10:58:32 +0000 http://www.deadlybloodyserious.com/2007/09/django-vs-feedparser-on-dates/#comment-2128 garth, utc should be pytz.UTC garth, utc should be pytz.UTC

]]> By: garth http://www.deadlybloodyserious.com/2007/09/feedparser-v-django/#comment-2124 garth Sun, 02 Sep 2007 02:50:43 +0000 http://www.deadlybloodyserious.com/2007/09/django-vs-feedparser-on-dates/#comment-2124 I'm gathering unique timestamps, and will try all these methods. Thanks! Anders, how is `utc` defined? I’m gathering unique timestamps, and will try all these methods. Thanks!

Anders, how is `utc` defined?

]]>