Monday, December 15, 2008

bluehackers.org

"It started with Arjen doing the last lightning talk at OSDC 2008… a
quick show of hands on who else had dealt with (or was dealing with)
depression. Everybody had a look around, and thus knew that they
weren’t alone. Afterwards, there was more positive feedback which
continued over email in the days that followed. Someone suggested
starting a group, and the same day bluehackers.org was born.

The objective of this initiative is to make visible that there are
many fellow geeks among us who are intimately familiar with
depression, anxiety, or bipolar disorder. It helps to know you’re not
alone. And it’s not because we’re geeks, but because we’re human. The
Australian BeyondBlue site is of course an excellent resource, but,
because geeks have a specific work environment, there are also
particular challenges in dealing with these issues, and that’s where
we feel our group can help with additional insights, tips, and posts
from others with experience.

Using the logo, we can also make the topic visible at meetings and
conferences around the world, ensuring that indeed no geek need feel
alone in this, or feel unsupported. They can simply look around and
see. Anybody will be able to show their support and understanding, in
a kind and non-intrusive manner."

...a repeated message from Arjen Lentz

Friday, December 5, 2008

OSDC liveblogging: OLPC in Australia

Presentation by Pia Waugh, OLPC
pia@olpcfriends.org

To clarify:
- project is not the KR thing
- not paid by OLPC, voluntary talk

Great FOSS project
Opportunity to create a regional approach
- connected classroom
- shared resources
- supplementing poorer countries
- do have great need in our regions, despite relative advantage overall
- create community of educators etc

Imagine if many people's education were based on an open source platform

Child takes a sense of ownership / responsibility for OLPC device
Better for each to have their own rather than to share for this reason
Target group is 6-12 years old

Important tenet is connectivity
- Can visualise mesh network of who is nearby
- FOSS, building blocks, building blocks for creativity

Some of the technology
----------------------

Antennae takes 5V in, can be used independently for mesh range estension
School server version
- jabber, other pooling tech, security tech, alerts if laptops leaves it will shut down unless it can reconnect to school server after some period of time
- backs up other laptops, allows teacher monitoring etc
- mic jack doubles as voltmeter

Regional Vision
---------------
- Great need in the region
- Build a strong developer and educator community
- e.g. indigenous communities and children in impoverished areas

5000 laptops donated to regional project, trials include many pacific island communities
Providing sat networking to some of the most remote areas
Some potential to travel with OLPC and contribute in the community, volunteers needed
wiki.laptop.org/go/OLPC_Oceania

Consideration of cultural values
--------------------------------

When rolling out, most pacific culture has healthy respect for teacher as the leader

Try to make sure teachers get the device early, extensive trainining, teacher is the person that gives the children the resource rather than from random externals.

Fit with what groups are already trying to achieve with their schools

Australia's first trial is happening
------------------------------------

A single laptop by itself doesn't have so much value, need classroom environment
e.g. spelling tests, collaborative writing activity

World's first remote-connected classroom. That is, the local teacher is connected to remote classrooms, video chat etc.

Video demo of remote classroom showing flashcard spelling exercises with student

Observed benefits to children
-----------------------------
Educational:
- Literacy improvements
- ICT skills -- typing, programming, games
- Maths
- Collaboration

Behavioural:
- Engagement of troubled children
- Reduced truancy

Example of Python shell where you just run demo code then change it yourself as an exercise

Example of a particular troubled kid from a very disadvantaged background who engaged through the laptop environment, partly because he could engage with his interest area, partly because the laptop was his, that he could use according to his own preferences and progress in his own time.

Benefits of a regional process:
-------------------------------
Funding, support, access to volunteers, knowledge xfer

OSDC liveblogging: Your code sucks and I hate you

Your code sucks and I hate you

First time patch submitted to OS project:
- nerve-wracking experience, judgement
- having people look at your code, give feedback etc is very powerful and important, best tool we have to get better as programmers

What's a review?
- Anyone even just talking about code
- A few types
- Involuntary review
- Random inspection
- Post-merge
- Pre-trunk (really good way of working, how bzr works)
- Pre-merge is the new hotness
- Enforces conformity (style consistent, ensures quality, enhance clarity)
- Moves from 1 person who understands code to 2 people

A few things in code review than can cause social problems
- Being told your entire approach is wrong
- Better to have a conversation before coding starts
- While you're there, totally do something else also
* need to be allowed to fix small incremental issues
* unclear outcomes (ie "this is bad", not what about it is bad)
- Name-calling is bad, doesn't help, talk about code not people
- Confusing personal preference with objective worth
- Filibustering -- what happens when you are outnumbered, but can win an argument by talking lots
- Lag is bad, reviews need to be fast, lag builds up
- Need to know who are the reviewers

Good things (how to do good reviews)
- Specific feedback
- Be thankful for code submits
- Ask questions where you don't understand stuff
- Before you start coding, talk to someone about it. Anyone.
- Decide to who gets the final say to head off future conflicts

Questions:
- Worst have experienced professionally is "that's a stupid idea"
- Language barriers can be an issue
- What happens if there are not enough developers for review?
- Generally in favour of doing code reviews in public or opt-out forum
- Be specific regarding thanking people also i.e. "I'm looking forward to using this feature"

OSDC liveblogging: Natural language generation in weather forecasting

Yesterday, I was unwell so I didn't keep up with my presentation blogging. Here is my presentation on Natural language generation in weather forecasting.

OSDC liveblogging: Legal Issues in OS

Brendan Scott, OS lawyer based in Sydney. (opensourcelaw.biz)
[n.b. this is particular to Australian law]

* Some analogy about driving to a location with a bit of suboptimal route planning. Many decisions along the way.

* When engaging a lawyer, this is like reading a map before setting out
* Not always necessary / justified, this depends on a bunch of things
- familiar territory?
- time and cost?

Hypothetical business growth path:
- start
- build it up
- sell it

Setting up an incorporated vehicle makes things (significantly) easier
The word "partnership" has a specific legal meaning, including:
- you are personally liable for your partners actions in the business
- best to avoid unless want to be in this specific relationship

How would you react to someone taking code from a project you are working on?
- If you have good record-keeping, can sort out rights relatively easily, even if you don't have a copyright record. Just need records of activity and can bootstrap copyright attribution

Contracts take effect at the time of signing

The conduct of a negotiation limit the things you can say in the future to try and vary the contract -- i.e. major points have been ventilated. Difficult to bring lawyer in at this stage to make nontrivial changes.

What else might a lawyer do for you?
- Dispute Resolution. Often a case of identifying what the problem is and identifying win-win or least-loss outcomes
- Sometimes knowledge of the law is extremely relevant

OSDC liveblogging: The State of Python

Presentation by Anthony Baxter

* 2.5.2 changes came from App Engine
* The Big News
- 3.0 is done
- .X releases are backward-compatible
- 2.6.1 almost out
- tinyurl.com/python3

IronPython continues on merry way
* Really fast
* Visual Studio is great
* First major bit of software with a decent license (pretty much Berkely)
* Targets 2.5
* Mostly works on Mono

Upgrade code:

* Take old code
* Port to 2.6
* Use -3 flag to show warnings
* Fix those
* Then run 2to3 -- rewrites code to 3 compat
* django is out for 3

Demo of upgrade to small app and highlight some specific changes
* print is a function
* print("foo", end="")
* __repr__ --> repr builtin, backticks gone

What's the future?
------------------
2.6.1 soon, 2.5 final not far off
3.0.1 a few months away

d.items() returns a dictionary view

If you want to learn python, learn in Python 3. New book coming out soon from someone who has been tracking python 3 for some time.

Thursday, December 4, 2008

OSDC liveblogging: Bazaar (VCS)

Gentle introduction to version control using Bazaar
Michael Hudson from Canonical

What is Bazaar?
Distributed VCS
No privileged sentral location
aims to be safe, friendly, free and fast

'bzr init'
'bzr add'
'bzr ci -m "initiam import"'

'bzr push' uploads or updates branch data on the server
'bzr branch' - get a copy of a remote branch
'bzr pull'
'bzr merge'

Some concepts
-------------

WorkingTree - collection of version controlled files and directories
Revision - A snapshot of a working tree and the fundamental object in bazaar
Branch - An ordered series of revisions
Repository - A place where revisions are stored
Commit - create a new snapshot/revision
Delta - the difference between two revisions
Merge - determining the outstanding revisions in one branch and applying them to another

* Can run a centralised model, then works quite like SVN

* Or, can run semi-centralised
- Mainline or trunk reflects current mainline
- New features developed in own branch
- Patch Queue Manager (PWM) can enforce this / guard a branch

* Full distributed
- Developers maintain their own branches
- Share / Merge p2p
- Can still use PQM to maintain a test-suite protected mainline branch

* Mixed-mode obviously possible

Advantages
----------
Don't need to give people commit rights to trunk to allow them to contribute (can send their revisions to other developers directly via 'bzr send')

A Bit About Launchpad
---------------------
A web-based tool to support open source software development
Designed to scale

Steps:
* Register launchpad id
* Register SSH keys
* Create branches, push/pull

Get some things for free:
* Server maintenance
* Web view of branch
* Branch notifications (commit mails)
* Simple access control through teams
* Support for code reviews
* Has email interface

Wednesday, December 3, 2008

OSDC Liveblogging: Openspatial software overview

------------------------------------
Generally about Open Projects:
- Are there procedures to contribute code?
- i.e. how useful is the project really

------------------------------------
Content Servers
------------------------------------
OSGeo
Mapserver, ,Mapbender, WMS, WFS, WCS

Simple features for sql (SFSQL)
SQL Multi Media Extensions

PostGIS (e.g. used to capture MARS ROVER data, rocket science, important)

GML, SML, GeoRSS, etc
GeoServer is a fully features j2ee geospatial data server

UMN MapServerr, original mover & shaker, high performance

deegree (with two e's) web services is quite comprehensive

GeoNetwork is good for discovering data, is being funded by an Aus govt group

------------------------------------
Applications
------------------------------------

JUMP / OpenJUMP (closed development)
GRASS running since 1982
QGIS
gvSig, open-source funded with 30mil euros public money
uDig, Eclipse RPC based
jGrass

------------------------------------
Thin clients
------------------------------------
OpenLayers is best bet to plug into variety of web-based mapping tools (important)
MapBender


Next year: FOSS4G 2009, Sydney, 500-700 delegates, yearly gathering of the tribes

In response to query regarding gridded data, suggested:
OSSIM: Gridded
GRASS has raster support
GAL+ODR, one of which has raster formats

OSDC liveblogging, Unittest: An under-appreciated gem

Unittest: An under-appreciated gem
Andrew Bennetts

What makes a good unit test suite?
- readable: intent and implementation is clear
- reliable: pass/fail when it shoud
- usable: easy to run, fast, debuggable

Very quick overview of unittest:
- Subclass unittest.TestCase
- def setUp(self): foo
- def a bunch of methods (will all be test cases)
- single unit test is represented by a TestCase instance
- then run tearDown
- report the result to a TestResult object

TestResult: passed to methods, record success details
TestLoader: turns test methods int TestCase subclasses into test cases
something else: is glue

Why is this good?
- Standard, most other frameworks will interoperate with it, batts included
- Not Python-centric, xUnit implementation, general programmer awareness
- Encourages good structure in tests
- By default, tests are isolated from eachother
- Easy to reuse a test fixture definition between multiple tests (reuse setUp and tearDown)
- Group tests by common needs
- test cases have explicit names, so clearer reporting, can be specific about what failed & where

Unittest is pretty easy to extend.
- extensions have a good chance of being fairly compatible with other test frameworks
- examined addCleanup logic, add cleanup code immediately after, e.g., adding a lock
- (Bazaar extension) multiply_test_suite_by_scenarios is a function that takes a test suite and a list of scenarios

A couple of extra libs worth knowing about
- launchpad.net/testtools: miscellaneous extensions from various projects
- launchpad.net/subunit: runs unit tests in separate processes to support test isolation
- launchpad.net/testresources: is a library to manage the initialisation and lifetime of expensive text fixtures

Some bits are bad:
- no standard tool to load and invoke a test suite
- parts of the API could be better (lots of 3rd party stuff needed)
- set of built-in assertions is a bit small
- documentation in the std lib is a bit weak

- fundamentally quite capable, pretty easy to usefully extend, would be great to get extensions back into std lib.
- launchpad.net/pyunit-friends

OSDC liveblogging - my GAE+ExtJS presentation

I think it went okay. My presentation covers and introduction to these technologies.

OSDC liveblogging -- Keynote by Chris DeBona

Chris DeBona oversees open-source license compliance at Google CA.

This presentation talks about how useful OS is to Google. Starts presentation with hardware history of Google, showing Linux ancestry. Current systems are big rack systems, but still running customised linux software with heaps of OS libraries involved.

Google has found about 3.5 billion lines of OS code in unique files, which is more than any of the big players individually have written.

In 2000, asked 10k sourceforge developers about their motivations:
* Intellectual curiosity / stimulation
* skills improvement
* work needs
* non-work functionality
* professional status
* reputation building
* distrust / dislike of proprietary software

Means that OSS developers understand OSS licenses

How does Google use open source?
* Linux kernel
* Apache
* Languages & compilers
* Engineers who run linux (Goobuntu)

Also, when releasing code:
* Android, Gears, apps

Does this to maintain independence from external software company
Allows you as a software company to do something out of the ordinary without showing your hand / without external dependencies

How does Google take part?
--------------------------

8.5 million lines of code released in Android
Release about 1 million lines of code outside of that
Important for engineers to be able to interact with peers in the community, not get out of touch
Provides good code hosting with version control, issue tracking, wiki
Hires some people directly because they are seen as important
Sponsors GSOC -- create new OS developers

Questions
---------

Why didn't Google create their own open source license?
* Don't trust licenses that arent's used much
* Real combination problems when integrating projects
* Not everyone trusts Google, so using well-understood licenses is good for this
* Obvious what Google means by using a license as people are familiar with them
* Developers don't need to understand new legal terms, familiar with known licenses
* Uses Apache a lot because it has specific things to say about patents (personal comment -- helps avoid patent issues -- not exactly sure how but seems to provide some end-user protection against patent infringement through using the software)

Now running out of battery, so posting now...

OSDC08 Day One

Yesterday (Tuesday 2 Dec, Sydney AUS) was day one of the OSDC 08 conference. It began with a Google Hackathon, where a few Googlers presented some of their latest technologies. These includes both open source projects which they sponsor (or at least contribute developer time) and closed-source but free-access systems. These systems were:
* OpenSocial
* Something about an open map layer system
* AppEngine

This was pretty neat, and I successfully copied-and-pasted my way to a functioning OpenSocial application which would allow me to give my friends virtual acorns in a variety of social network containers. Google make available a number of things to make this easier, including comprehensive examples, a number of free hosting solutions and a sandbox environment for Orkut to deploy test applications.

I haven't been active in this area of development before, but a great deal of the complexity of managing such an application has been stripped away to allow any potential developer to concentrate on application functionality instead of systems maintenance.

Following this, I visited the Google offices as a part of the Sydney Python User's Group meeting. At this meetup I presented on the topic of AI in Python. At this stage, I prefer not to make my slides web-available but would be happy to supply them via email to anyone at the presentation or who would like to engage on the topic offline. Since AI is such a huge area, with so many potential audiences, there is a lot of ground for misinterpretation of a slide or bullet point if taken out of context. I had great fun however, and greatly enjoyed catching up with everyone at the pub afterwards.

Props to Christoph, Mark, Alan, Richard, Mark, Matt, Alex, Joel and everyone else that I met whose names I can't right now recall. Thanks for making my stay a nice one!

-Tennessee

Thursday, November 27, 2008

IT is not like upgrading a car, like designing a house

I would contend that IT problems tend to be organisational problems rather than technology problems. Upgrading your IT is not like upgrading your car, where you just get better performance from your wheels, accelerator and brake. Rather, upgrading your IT is like designing a house. Yes, new houses are often built from better components, but you still need good design that meets your particular needs. To some extent you can buy off the plan, but the result is never as good as working closely with the architect.

Thursday, November 13, 2008

Book Meme

INSTRUCTIONS:

* Grab the nearest book.
* Open it to page 56.
* Find the fifth sentence.
* Post the text of the sentence in your journal along with these instructions.
* Don’t dig for your favorite book, the cool book, or the intellectual one: pick the CLOSEST.

"Not only is the flowering of global science and technology not an exclusively Western-led phenomenon, there were major global advances in the world that involved extensive international encounters away from Europe".

This week finds me reading "Identity and Violence" by Amartya Sen (subtitled "The Illusion of Destiny"). Last week would have found me reading Terry Pratchett, which probably would have made for better blogvertainment...

Cheers,
-T

Wednesday, November 12, 2008

Mashup idea: library catalague with book reviews

I'd like to see a really good book review site that could talk to my local library catalogue system, so that I could check out what highly-reviewed books my library currently has available for loan, and perhaps even use it at the basis for purchasing requests. Just another way that Things Could be Better! :)

-T

Thursday, November 6, 2008

Pythonista, pythoneer... pythonite

Another noun for describing a python afficionado. See http://www.wordinfo.info/words/index/info/view_unit/1096 .

As in socialite, canaanite.

Also related to the rare substance Pythonium, the pure essence of Python in mineral form. Pythonite is a substance created from Pythonium. Consuming Pythonite (or refined Pythonium) can lead to greatly enhanced productivity. Hence the somewhat ambiguous term Pythonite may refer either to a Python afficionado or the mineral substance formed when Pythonium is present in the natural environment.

Can react energetically with Perlite despite its proximity on the programmatic table of elements.

-T

Wednesday, November 5, 2008

More on soft exceptions

I had a real-world problem just now which I chose to tackle using soft exceptions. I have a series of rules which transform a data object. Depending on the state of the data object, not all the rules are applicable. Also, depending on the rule, it may or may not be critical to have the rule applied to the data object.

While pursuing another task, I caused an exception inside a method which applied one of these rules. Rather than simply track back the exception and add more handling to the code, I decided to *also* wrap this rule in a soft exception handler. In operations, it would be far better to simply not apply the rule than to cease processing and crash, yet the rule logic is written such that any error causes a program fault.

Ideally, I would implement the soft exception handling using a logger level which could be applied application-wide, but that is a bigger task than I wanted to tackle in 10 minutes!

So, I wrote:


exceptionMode = "Soft"

try:
originalState = object.state
// Apply rule logic

except:
object.state = originalState

if exceptionMode == "Soft":
print "NEW WARNING -- generated in MyClass.methodName
print "Methodname failed. Original state is being used."
traceback.print_exc()
print "END WARNING"

else:
raise

return successFlag


This worked beautifully! I can check the code in with soft exceptions turned on, so that in operations any bugs here will only result in a rule failure, not a complete process failure. However, I still have access to the full stack trace through logging and can also set the exception mode to "Hard" (or anything that's not "Soft") and have 'brittle' behaviour for testing.

Score 1 for soft exceptions!

Clearly, it would be ideal to log instead of print. It would be good to have have the following logging controls:
* Hard or soft exception handling
* A logging level which would actually bring up user warning dialogs
* A variety of logging detail levels

Cheers,
-T

Monday, November 3, 2008

Meme: Top Five Software Possibilities

Here's a quick meme... List the top five things that (you believe) could be feasibly achieved with a software R&D investment in the order of $30 million. Please list a time frame for any particularly big ideas. They could be your top five moneymakers, top five in terms of benefit for society, or anything -- it's YOUR top five :)

1. Flawless text-to-speech rendering. Could be done in four years, will take 15 otherwise.
2. Driverless cars. Could be done in six years, will take 40 otherwise.
3. Complete automation of the tax system. Maybe six years?
4. Creation of fully anonymous, publicly available medical research datasets.
5. Create an artificial mind. Could be done in ten years, but may never be done.

What are your top five?

-Tennessee

Sunday, November 2, 2008

My project: GAE-JSON

URL: http://code.google.com/p/gae-json/

The project is really not ready for public consumption, but I have noticed another project called gae-json-rest which is turning up on Google search. So that others can find my project by searching for it, I just needed to create a couple of links to it so that it will be available. Not, of course, that I'm being competitive -- but if I don't put some links up then there will be a lot of confused visitors to the gae-json-rest project!

If anyone feels very enthusiastic about the project, there *are* a couple of functional examples which anyone could install on their system, but they will need some guidance first. By the time OSDC08 comes around, however, I intend to have turned this around into something which is really genuinely user-friendly. I should add to the list of my goals:
* Create a hosted demo
* Put together a wizard / walkthrough
* Create some documentation

However, goals at the moment will still be those relating to functionality...

-Tennessee

Friday, October 31, 2008

This close to a datastore --> json server on google apps engine

Phew ... a good afternoon's hacking.

I am *this close* to completing a very basic prototype of a json server for google apps engine. The idea is that you visit a URL which contains the query you want to evaluate (hence my previous post) and get back a JSON encoding of the data. Obviously, that's for requests only.

However, what it *means* is that you can then write an AJAX client which can interface directly to a GAE data store. And *that* is a good thing.

Sure, it's about as functional as a car with no steering wheel, but in principle it works. I have achieved the following goals:
* Store some data in the data store
* Retrieve it with a GQL query
* Write a web server which decodes a query from the URL in hex
* Use that to retrieve said data from the query store
* Render the data to a web page in HTML
* Render the a json string to the web page (or json-looking, anyway)

The following goals remain:
* Write a version which emits the json string with appropriate mime type
* Write a javascript client which retrieves the data and renders it in a web page
* Put up some vaguely comprehensible web pages explaining the thing and demonstrating its use
* Make the store/retrieval more general by:
-- Not relying on a specific database structure
-- Going beyond string-only data
-- Do something about INSERT queries

I'm having some fun with this. Anyone who is interested, let me know and I can keep you posted of progress. I think it should be really useful.

Efficient encoding of URL arguments

I would like to have URLs which look like:
"http://web.host/queryhandler/Tnidf3445ssx34" or something similar. That last part should be some reasonably space-efficient encoding of something like "SELECT * from FOO where CONDITION"

I have found I can get part-way using Python's encode and decode functions, e.g.
encoded = "SELECT * from FOO where CONDITION".encode("hex")
decoded = encoded.decode("hex")

However, the resulting values are still a bit long. This method quite nearly removes nonprintable characters, spaces etc from the URL. However, I have this nagging feeling that I could be doing some compression along the way to make the encoded part shorter. For example, HEX uses only part of the alphanumeric space. I'd like to use the digits 0-9 and letters a-z and A-Z ideally.

Let's just make this clear -- for no very good reason that I can justify, I want to encode a query as the shortest possible string using printable, alphanumeric characters (and no spaces) and be able to simply decode the string also.

"The Internet" has told me about md5 hashing, but I need a two-way function here. The help on string.encode in Python simply says I can specify the name of any encoder, but I can't find a list of the default encoders available.

Any ideas, internet?

Cheers and thanks in advance,
-T

Thursday, October 30, 2008

OSDC 2008 pre-conference activities

I notice that the OSDC conference days have dropped from the 1st - 5th
to the 2nd to the 5th as the conference schedule has been firming up.
I find myself now with a free day on the 1st. I wonder if there are
any Pythonistas in Sydney who would care to catch up on that day for a
discussion group, BoF sessions or anything else? Looking at the
program, it seems to have fewer of such events than past conferences.
There are some available slots for these sessions, but if anyone wants
to get together for some more in-depth discussions, I'd be up for it.

I'd be very interesting in informal sessions on the following topics:
* Open source in government
* Weather forecasting
* AJAX development
* Artificial intelligence
* Advanced Python skills
* Open Source & Project Management

I have also posted this message to various Australian Python mailing lists. So far there are two takers for some activities on the Monday. If interested parties can email me, I will see if there are enough takers to organise some small-group discussions on topics of interest. So far the takers are 1 for Open Source in Government and another for AI.

Regards,
-Tennessee Leeuwenburg

Wednesday, October 29, 2008

Automatic text generation is live!

Hi all,

The system which I have been working on for three years has just gone live. My component of this system generates text forecasts automatically, based on gridded forecast data which is initialised from model conditions and then manipulated by weather forecasters. Any particular text forecast may also be edited by the forecasters after generation, but the bulk of the forecasts are published without forecaster intervention in the text.

Examples include the Melbourne forecast,
http://www.bom.gov.au/products/IDV10450.shtml

A number of other Victorian town forecasts, Victorian district forecasts and an expanded set of Precis and Town forecasts are also produced. Also, a larger number of forecast days are included for many of these locations.

This page (http://www.bom.gov.au/nextgen/) describes some of the changes to service which come with the new forecast system.

Cheers,
-Tennessee

Friday, September 26, 2008

Code, wiki style

I've got this use case for editing code through a web interface, like a wiki. I thought I'd find out if anyone else has thought about this topic, and didn't see anything jump out at me when searching.

The situation is that maybe there's an open-source or perhaps just shared-development project in a workplace, hosted on the cloud. Rather than each person having an IDE set up for them or accessing a CVS repository, they just have a web interface.

Maybe the system requires logins to track changes, or in the first instance maybe it just allows unrestricted and un-tracked editing.

Does anyone know of such a system? Ideally, the server code would be written in Python (I'm trying to set up a Python-only technology stack for reasons of minimalism) but I would be interested in non-Python systems. I don't mind if the interface is fairly primitive to start with, but it would be neat if it did include some additional features such as syntax highlighting.

What do people think?

Cheers,
-T

Tuesday, September 16, 2008

search through code, search through comments

Dear lazyweb,

I would love it if I could search through my code using my IDE, with the following options:
* Search all text in files
* Search only comments
* Search only non-comment code

My current favourite IDE is Eclipse/PyDev, but I'm not sure where to lodge queries easily, so here's my message in a bottle for the relevant developers. God speed, little bottle!

Cheers,
-Tennessee

Monday, September 15, 2008

OSDC 2008 GAE + ExtJS code samples

This page contains links to demonstration applications developed for OSDC 2008, including:

* JSON server written in Python integrated with GAE and the Document Store
* Basic ExtJS form submission to Google Document Store

For GAE-JSON SVN server, visit: http://code.google.com/p/gae-json/

Wednesday, August 27, 2008

Soft exceptions

Something that I would like to consider for the future is an idea I call soft exceptions. They are probably industry standard already and I'm just the last to know, but here's the idea.

Many times I want to execute a block of code which does something, but not something important. If I'm writing some important piece of software, I don't want problems in unimportant code areas bubbling up to cause big problems in areas that *are* important. This can be managed by writing great code, having an entire bureacracy of test scripts and good defensive coding habits. Or you can just wrap the whole lot in a try statement and best practises be be blowed.

I've got this idea that there can be a middle ground -- a way of being able to protect your important code when it's live and being relied upon (other than by just not having errors happen) while not putting in kludge after kludge to keep the motor running.

The idea is, basically, that exception handling would be linked to the logging system, and that whether an exception was fatal or not would depend on the logging/debug level set.

For example, I might have a code loop which is something like the following:


while(True):
doSomethingReallyImportant()

try:
doSomethingNotVeryImportant()
except KnownFailureMode:
print "Sorry, there is no wheat in the hopper"
except:
log("Something not very important broke unexpectedly")

if logLevel = "soft":
log('soft exceptions, see traceback')
log(traceback.format_exc())
else:
raise


This code would
a) Protect the important code in the loop
b) Respond to known failure modes
c) Log everything
d) Re-raise unexcpected exceptions if in debug mode
e) Suppress unimportant things if in operational/critical mode

I could even imagine "fallback mode' operations which could be user-enabled which would enable hard exception handling....

Just some thoughts....

Friday, August 15, 2008

Melbourne-based Javascript experts

Hi lazyweb,

I was wondering if there are any Melbourne-based javascript experts who would be able to help me with a bit of learning. I could really benefit from being able to pick someone's brain for a few hours, with problems in front of me etc. I don't want a training course, just a kind of expert tutor for some assistance. I'd be happy to pay a fair hourly rate for the time spent of course. Google a.k.a. the Internets didn't really have any obvious promising leads, so I'm hoping this might find its way into the inbox of a real person who could help me out. God speed, message in a bottle!

Email me at tleeuwenburg@gmail.com.

Thanks,
-Tennessee

Friday, May 30, 2008

Web application design philosophy - HELP!?

Full disclosure: I started developing web apps like two days ago. This post is written as a n00b, not as a source of authority.

Not to be intimidated, I got out my slate and chalk, and began doing screenshot designs for a rich internet application (RIA) I want to write. I found myself some whizzbang technology, in this case Google Apps Engine and the Extjs javascript widget framework. They're both pretty easy to work with (so long as you don't need to do any debugging) and I managed to prototype myself some pages to do Neat Stuff.

I'm now faced with a difficult path to actually learning how to write a web application "properly". With a standalone GUI app, I'm pretty much fine with application design, breaking things up into an MVC model and then doing an object-oriented design of each bit. With a web application, it's not so clear how I should do things.

If I were designing something in static HTML, all the logic goes on the web server. The user clicks "load data" and the web server goes and produces a static HTML page showing the data loaded. The application brain is in the server, and the purpose of the HTML page is to render the user interface to the client.

It appears possible to basically do the complete opposite with javascript, and put basically all the processing logic into the javascript, treating the web server as just something to go fetch resources from.

So should the logic go?

I'm worried that if I pursue a heterogenous design, whereby the application logic is split between client and server, that I will then have an unsatisfactory level of coupling between the two. Should the database be the 'model', the web server be the 'controller' and the client be the 'view'?

Presumably, if I put heaps of logic into the javascript, then network overhead will be high but latency will be good? Whereas perhaps if I do it the other way, network overhead will be low but many more requests will be necessary, thus slowing application responsiveness?

Where do XML web services and SOAP requests come in? "Should" I be using these as an API, or should I just request data and perform my processing in javascript?

How do I write a web application such that it is reasonably 'pluggable' -- that is such that I could alter my choice of server, database or javascript extension library without killing myself?

Should I use something like Django's templating to control the GUI or should I put all the GUI design into my javascript? Should there be a javascript object representing the whole application which then loads the navbar, display and other widgets?

Should I do form processing on the web server, or in the javascript?

ARGHHH!

I think, maybe, the index page should be a trad "web page" kind of page, with links to the RIA pages.

-Tennessee

Wednesday, May 28, 2008

OSDC 2008

From 1 to 5 Dec, in Sydney, Australia, the Open Source Developer's Conference 2008 will be held (http://osdc.com.au/2008). I will, in all probability, be attending this. I would love to get in touch with any other Pythoneers who may be going. It would be fantastic to organise some group sessions and do some networking. Also, if there's anyone else who works for the Aus govt it would be good to hear about what kind of open source applications are getting a guernsey in your depts.

I thought it would be neat if we could find some leading Aussie python people and rope them into giving a panel talk on some topic, leading a discussion or some such.

Tuesday, May 20, 2008

Advanced Document Notes

I just started a Google App Engine trial and my goal is to prototype an application I've always wanted. If it works, I will even use it. It's really simple to explain -- I want to take notes against documents. Then, I want fulltext search through everything.

There are files. Some are public some are private. Let's assume they are all PDF articles that I want to write about.

There is a common document store and a private document store. Documents can be shared, and you can read eachother's notes! Yay! But some are private, because sharing them would be bad for some reason. So you have access to the global store (which is like a wiki) and the private store.

Okay, so you log in, and can upload a file. Yay! Then you take notes against it, which is stored in a notes file and related to the original file.

Then, you can have a display where the PDF is viewed on the left and the notes are viewed on the right.

So, you could share something like a PDF, text file, web page or even binary media file, and share comments on it. I envisage it as being great for the following uses:

1) An academic who wants to create a personal store of research notes
2) An academic who wants to collaborate on creating research notes, such as with fellow researchers or students
3) Anyone who wants to share comments about files.
4) A workplace where a group wants to share around a file for comment

I just hope I can find time to actually do this. I don't think it will be particularly hard, but neither do I have a lot of free time.

Anyway, cool! GAE!

I'm trying not to think of any additional functionality at this stage, but ideas keep popping into my head anyway.

-Tennessee

Sunday, March 23, 2008

Requirement: Taking notes against a document

Here's a tool that I want. It's simple, and clearly useful. I study, as do many people, and I think the use case goes further. I have documents, which I don't need to edit -- e.g. articles and papers. I need to take notes against these documents. Further, I would like to be able to search within the body of documents and notes. I want to be able to create an articles hom directory (~articles). I then want to be able to have my own subdirectory structure (~articles/subject). I then want to be able to create notes files that are associated with each article. Then, I want a search tool (like Beagle or Google Desktop) which will constrain itself to ~articles, and know about the association between notes files and articles. I want full-text search, with a link on the results page to both the article and the notes. That would be great!

-Tennessee

Wednesday, February 20, 2008

From my python interpreter

>>> a = 1,2
>>> a
(1, 2)
>>> a == 1,2
(False, 2)

Friday, January 25, 2008

Article Notification: The Impact of Component Modularity on Design Evolution: Evidence from the Software Industry

This paper interested me and I thought it might interest others.

Title: The Impact of Component Modularity on Design Evolution: Evidence from the Software Industry
Authors: Alan MacCormack, John Rusnak, and Carliss Y. Baldwin
URL: http://hbswk.hbs.edu/item/5831.html

Executive Summary (from the site):

What factors should influence the design of a complex system? And what is the impact of choices on both product and organizational performance? These issues are of particular importance in the field of software given how software is developed: Rarely do software projects start from scratch. The authors analyzed the evolution of a commercial software product from first release to its current design, looking specifically at 6 major versions released at varying periods over a 15-year period. These results have important implications for managers, highlighting the impact of design decisions made today on both the evolution and the maintainability of a design in subsequent years. Key concepts include:

* Data show strong support for the existence of a relationship between component modularity and design evolution.
* Tightly coupled components have a higher probability of survival as a design evolves compared with loosely coupled components.
* Tightly coupled components are also harder to augment, in that the mix of new components added in each version is significantly more modular than the legacy design.

Tuesday, January 22, 2008

comapping.com

I just found the most astoundingly useful web tool. There's a free 30-day trial and I highly, highly recommend that everyone take it for a spin. It is, essentially, an intuitive structured-note taking tool. It appears to create notes in a tree-based structure, with node types which correspond to projects, goals, tasks, notes, meeting agendas etc depending on your chosen template.

http://comapping.com/

Cheers,
-T

Thursday, January 3, 2008

Developing for userspace (linux-focused)

On great Python utility is virtual Python. It is a script which will take your system Python install and leverage that to create a Python binary in your home directory. The advantage is that you can then install all kinds of third-party libraries without (a) root permissions or (b) messing up your primary installation.

It can be used to create several different "homes" for Python, which could let you experiment with library version, install procedures or a variety of other things.

More people should develop for userspace.

I was recently trying to install C#, something which I would love to learn. On linux this means mono. On a standard operating environment without root priveleges, this means a *lot* of dependencies (due to old versions of Stuff) and a lot of compiling.

It would be great if RPM (and apt, or pick-your-favourite-installer) could handle the idea of a userspace package repository. Maybe they can and I don't know about it, but I've not seen any information on the topic. Things that get installed should have a userspace option. If I want to grab the latest gnome, mono and C# libraries into user-space without root priveleges, I should be able to. I should be crippled just because I'm not root. It doesn't seem beyond the capacity of linux to allow users to do this, since it already supports the userspace paradigm very well.

-Tennessee