Dr.K: News Machine

Having tried both methods of RSS to HTML - I have decided to use BuzzBoost from FeedBurner rather than RSS Include.

My reasons:

(i) RSSInclude only provides for 10 news items and BuzzBoost allows for unlimited items

(ii) BuzzBoost supports more in the way of CSS than RSSInclude and can be configured any way I choose.

Not that I am unhappy with the service that RSSInclude gives - just that as my feeds are coming off FeedBurner anyhow, and that I can customize the BuzzBoost code to give me more control over the look and feel of the News-Machine pages - means that it wins hands down.

For now ...

Tags: feedburner, buzzboost, rss, html, headlines, rssinclude, rsstools

Posted by drk at 04:51 PM | Permalink

Because my main blog moved host and URL this weekend I was re-assigning DNS for the hackershandbook domain and managed to totally break this weblog.

It now looks like its fixed - the problem stemmed from an incorrect URL pointing to the CSS used for this site.

Sorry about the problems - normal service has now been resumed.

If I've missed any links and anything is broken - let me know.

Posted by drk at 02:35 PM | Permalink

The other thing I have changed with the SEO News page for Tim is the way that RSS is presented

Up until now I've been using RSS Include to embed the RSS inside the webpages here - but the limitations of RSS Include are that there are only 10 headlines - not good when you are trying to track between 10-50 news items - especially as the Google News engine always hogs the top two items ...

So I'm trying the Feedburner Buzzboost RSS to HTML program instead.

I'll see how it works out - I miss the control over the look and feel of the RSS displayed inside RSS Include - but the 10 headline limitation means that right now BuzzBoost is better for displaying all the headlines that come up inside the KCU search group.

Feedback please - I will set up a comments page in the next few weeks - until then just email me.

Which looks best? RSSInclude or BuzzBoost?

Tags: rss, rssinclude, feedburner, feedburner buzzboost, buzzboost, news-machine

Posted by drk at 08:59 PM | Permalink

I've just made a new page for Tim Rendle so he can track the current news in syndicspace about his chosen topic.

If he chooses his keywords correctly - then the whole of syndicspace will be available for him to explore his chosen subject.

This is true - especially if he uses the tagsearch view for auto-discovery of new keywords on a regular basis.

Sometimes the web need human input after all!!

Then feeds it back to the KCU cluser unit (i.e. email me with changes!)

Used correctly it should enable Tim to track new keyword spaces in real time. (within the cycle of updates in the online systems being used for this experiment - which means a high degree of error. Google News - for example - sometimes doesn't update its news view for days - meaning the same cloudsearch view will appear for days on end).

The current keywords for Tim's KCU are: google optimization, keywords, primary keywords, secondary keywords, seo, search engine optimisation, website promotion, adsense, adwords, traffic analysis, pagerank, alexa, link analysis, google, search engine, web spiders, pay-per-click, ranking optimization and webmaster.

This should change - the autodiscovery feature of the tagsearch engine supplied by ZoomClouds means that if this system is monitored and feedback given - the KCU should stabilise very quickly and redundant keywords eliminated to make a more compact tagsearch view - and a more compact news search.

CAVEATS: I hate the way google insists on not letting me remove their "Top Stories" from the page/feed - it messes the whole thing up big time ...

Tags: news-machine, drk, rss, experimental, tim rendle google optimization, keywords, primary keywords, secondary keywords, seo, search engine optimisation, website promotion, adsense, traffic analysis, pagerank, alexa, link analysis, google, syndicspace, search engine, web spiders, pay-per-click, ranking optimization, webmaster

Posted by drk at 08:18 PM | Permalink

I've resuffled Space Science News and broken it into three pieces:

Space Science News: The emphasis is mainly on near earth technology and space exploration

astronaut, cassini, deep space network, earth observation, galaxy evolution explorer, geosynchronous orbit, hubble, huygens, iss, lunar exploration, mars exploration, mars global surveyor, nasa, satellite, soyuz, space probe, space shuttle, space tourism, spirit rover, spitzer

Solar System News: Science and Astronomy solely focussed on the solar system

asteroids, solar system, venus, comets, solar storms, earth, ,jupiter, sunspots, magnetic storms, mars, mercury, titan, meteorites, near earth objects, neptune, planets, pluto, saturn, solar flares, uranus

Astronomy News: Theoretical and Cosmological type stuff

astrobiology, astrophysics, big bang, binary stars, black holes, chandra, cosmology, dark matter, galaxies, gamma ray bursts, nebulae, nova, observatory, pulsars, quasars, radio astronomy, seti, star clusters, stars, supervova.

The system isn't perfect but its a good chance to see how it scales - the question is - how many keyword cluster units do you need to completely cover a field like astronomy?

Posted by drk at 06:04 PM | Permalink

I've added an extra page for a mate who designs embedded systems - currently it tracks the following keywords:

PIC, microchip, ethernet, embedded web servers, web on a chip, embedded controllers, micro controllers, embedded linux, tiny linux, php, mysql, damn small linux, redhat, embedded home control, embedded house control, remote control - but I think this will get updated quite soon - the TagCloud suggests a lot more keywords that could be used and there is redundancy at the moment.

Tags: PIC, microchip, ethernet, embedded web servers, web on a chip, embedded controllers, micro controllers, embedded linux, tiny linux, php, mysql, damn small linux, redhat, embedded home control, embedded house control, remote control, cybatek

Posted by drk at 06:04 PM | Permalink

Because of the limits of Google News - 20 keywords in a single account - I've bundled keywords into top-pevel categories that approximate projects I'm working on - then for shorthand named them "Keyword Cluster Units".

Each KCU is made up of 1-20 keywords which are inserted (by hand) into Google News - the resulting RSS news feed is then available for further processing anywhere.

Example: Current Obsessions - the keyword unit cluster that tracks news which I am interested in blogging about.

cleanfeed, hackers, privacy-internet, virus, censorship-internet, microsoft-security, copyright, drm, riaa, mpaa, piracy-internet, piracy-sea, malware, blacklists, smartfilter, dmca, spyware, exploit, security-internet, censorware.

The major problem is that each KCU has to be maintained by hand - and everytime the KCU is updated then the Google News RSS feed URL also changes - so even minor changes are a pain and a major re-arrangement is a major pain.

The other problem is that Google News ties everyuser to an email address - so for the 5 current KCUs I need 5 email addresses - login into Google News - make the changes and then cut and paste the changed URL.

I would also like to get rid of the "Top Stories" they clutter the feeds with irrelevant duplicate stories ...

It would be great if Google News supported (a) more than 20 keywords for an account, and (b) a method of clustering keywords into categories and then feeding them to category led RSS feeds.

Right now the situation works well for a few Google News feeds - they all come out marked Google News - but by the time I've republished in FeedBurner and re-written the Feed name with the KCU name - the feeds end up marked with both the category name of the keyword cluster - and the keyword itself as a category within the feed.

It works - but its kludgy and maintenance is a pain - and I still haven't folded the stuff from BlogSpace into SyndicSpace yet ..... and all my attempts at online aggregation have led to nothing but online aggravation.

Tags: rss rojo feedburner google news atom Syndication aggregation tagging filtering clustering

Posted by drk at 06:02 PM | Permalink

I'm still on the holy grail of building the mother of all news filtering engines out of online sources - the story so far.

Google News: searches 4500 news sources - can be adjusted to search by keyword(s) - limit of 20 keyword searches per userid - userid tied to email address - RSS available

Problem - not enough keywords available

Solution: Multiple Google News groups with multiple keyword cluster units - each account clusters keywords according to a category - see the example feeds under keyword clusters.

Next up is tagging the keyword cluster units somehow - I originally wanted to aggregate using rojo into one feed - but no! rojo barfs when i tried to give it more than one raw google news feed.

Solution: Feed each filtered google newsfeed into feedburner and reburn the feed name and tag it with the keyword cluster name - now each feed is named with the keyword cluster unit name and and each post is tagged with the keyword as category!!

This is neat!! - Fed into RSSOwl, Omea Feedreader or rojo I can use keyword cluster name as top-level category and still see the keyword as a subcategory.

e.g. keyword cluster name = Extreme Tales News, category - skateboarding or keyword cluster name = Space Science and category = nasa - and so on.

I know at a glance which cluster (category) the story has come from and also which keyword triggered the story from within Google News.

Now the problems begin.

I have 5 keyword cluster units feeding into 5 feedburner feeds to provide the info I want - but I want to aggregate them into one feed - RSS or Atom - to pass to wherever I want.

Things get screwed from here in.when I subsrcibe to the rojo RSS for "all your stories" it strips out all the carefully placed keyword cluster unit names (blog names) information - and also hoses the category information.

That's two levels of categorisation gone!! I'll look at the raw xml sometime and try and figure out what rojo is doing with my category information - sure as hell Omea can't see it.

In Omea reader I can see the sub-categories of the rojo feeds - but ONLY if I define a feedreader sub-category for that feed. Otherwise nada. This is - as they say - not good.

The last thing I want to do is keep updating Omea categories every time I change a keyword and the idea of putting a 100 categories into Omea Reader just becasuse rojo has stripped them out is horrible.

So while the feedreader feeds tag the stories with the keyword as category - allowing me to see at a glance which keyword it came from - rojo breaks this totally.

Worse yet - RSSOwl will have nothing to do with republished rojo feeds.

So the quest continues - I'll try and update this here and keep it separate from my main blog - its all experimental anyway - and also blog roll the feeds I am playing - maybe better minds than mine know how to solve the problems!!

Tags: rss rojo feedburner google news atom Syndication aggregation tagging filtering clustering

Posted by drk at 06:04 PM | Permalink