http://www.wordle.net/gallery/wrdl/921427/Wordle_Core_Source_Code

Also, if anyone is still interested, there’s a command-line version of Wordle available here:

http://www.alphaworks.ibm.com/tech/wordcloud

I think the main thing you’re measuring here (aside from language verbosity which you already mentioned) is just how well the variables and methods are named. Even then, a good architecture might have everything namespaced inside a very important module. If the module is cleanly decoupled it may not appear high in the word count at all even if it’s the most important word. Low-level methods and local variables will be favored, which (especially the latter) are the easiest part of the code to understand and have no bearing at all on whether the architecture is good.

As far as trading in standard language keywords for DSLs, that always sounds good in theory, but what you are doing are trading standard language semantics for something domain-specific which may or may not be confusing, leaky, or non-obvious. With standard language semantics you have a huge ready-made pool of talent. For a DSL to shine the domain should naturally lend itself to a formal description, rather than shoehorning it in for the sake of aesthetics.

When I think of what goes into a good architecture, neither over-engineered, or under-engineered, well-tested, de-coupled, maintainable and performant, I have a hard time seeing where optimizing the word counts would not push out some more important design criterion. I think it’s a good exercise to create these word (they’re not tag) clouds and make observations, but the moment we start using it as a metric then people will optimize for it, which I don’t think would be win per se.

I concatenated the code in a similar fashion (aggregated using ack) and just pasted code straight from emacs. It was around 64K lines for each example and I had no trouble. I user Safari 4.0 and Java 5 (for the applet).

I thought about using the advanced tools to mark in different colours the multiple domains -technical and business in this case- but was too lazy to actually do this myself.

How did you do it?

I started by concatenating all the code files together[1] and pasting the results into the Wordle “Paste a bunch of words here”.

I eventually discovered the Wordle Advanced Tools and wrote a monster command-line[2] to find the top 200 words in a codebase.

In my sample of two:
* A Rails codebase (just the “app” folder) — relatively domainy, relatively few “programmer” words like “def”
* The major section of a Java codebase: almost no domain words at all in the top 50

I believe the result is an accurate reflection with how generally expressive the code is in each codebase. The Java one makes heavy use of “String” and “get” and other machinery words.

–
[1] Concatenate all Java files together

find . -name '*.java' | xargs cat


[2] Identify the 200 most-used words in across all Java files (treating camelCase as two words)

find . -name '*.java' | xargs cat | tr -s '[[:blank:][:cntrl:][:punct:]]' '\n' | grep '[[:alpha:]]' | sed s/'\([[:lower:]]\)\([[:upper:]]\)'/'\1 \2'/g | tr '[[:blank:]]' '\n' | sort | uniq -c | sort -n | sed s/'^\s*\([0-9]*\) \(.*\)$'/'\2:\1'/g | tail -n 200


There are -at least- two different ways to use the cloud. The first one is what I did, to try to get the ratio between noise caused by the language and signal in the code. This is useful to show how DSLs attack noise, for example. If you remove the keywords you won’t find it.

The other approach I see is to do what you guys said and filter out known keywords and maybe other known entities - framework classes, for example. This is extremely useful in a different way: it tells you what the ubiquitous language looks like. Using this approach you can even try to study how Bounded Contexts are used in an application and how cohesive a system is.

I am working right now in trying to write something about those multiple uses. The hardest part is to find some good public code bases to explore.

@Ivan
I’m writing code too, maybe we should try to do something together. Still need some prototyping time alone, though, but I’m following your repo.

Great idea: highly useful, effective and simple application for word tags.

Congratulations!