Jaoo 2009
Table of Contents
- 1 Monday, Oct 5, 2009
-
- 1.1 Keynote: Scaling up Agility: The architected agile approach
- 1.2 Working effectively with legacy code
- 1.3 Evolving the key/value programming model to a higher level
- 1.4 Techniques that still work no matter how hard we try to forget them
- 1.5 Making use of patterns
- 1.6 xUnit test patterns: Refactor test code to improve ROI
-
- 2 Tuesday, Oct 6, 2009
- 3 Wednesday, Oct 7, 2009
1 Monday, Oct 5, 2009
1.1 Keynote: Scaling up Agility: The architected agile approach
- Barry Boem
- 100 years to work out cars, but software so much more rapidly paced
- The auditors manifesto rather than the agile manifesto
- Using risk to balance
- Situation aware
- Milestone where people can come together
- Incremental releases
- Keep steady development team
- Don’t switch tasks all too often
1.2 Working effectively with legacy code
- Michael Feathers
- Talk based on Working effective with legacy code book
- Talk reflects learning’s from when book was published in 2004
- No tools for brownfield development except for refactoring tools
- Global variables not so bad anymore
- Not in the sense that everyone needs them, but used across many classes
- Global variables aren’t the problem, global state is
- Global variables => separation of concerns
- Globals with individual components aren’t that bad
- Repository: Collects global state in one place
- Factory: Should be stateless
- Writing tests, go wide for coverage
Write tests for higher level classes, if possible, before making
changes to lower levels. - Seam
- Altering behavior without editing in-place
- Subclassing original class
- Making use of C++ preprocessor
- Refactor towards explicitness
- Break dependencies. Testability => better design
- Excessive setup typically means excessive dependencies
- Slow tests indicate insufficient granularity of code (high coupling)
- Testing is a way of understanding code
- Altering behavior without editing in-place
- Type cruft
- Go framework happy
When using lot of libraries, these libraries have to be used
in tests as well (e.g., currency library, SQL spatial
library). Not taking these dependencies makes code easier to
work with.
- Go framework happy
- Tell, don’t ask
- Leads to lower coupled, more testable systems
- Tell objects to do something, don’t act as an intermediary
- Functional programming languages
- Pure functional code is easier to understand
=> good separation of concerns
=> powerful type systems - Seems with functional languages aren’t quite mature.
Override a method by replacing a method isn’t really
possible. Although functions can be passed to function
- Pure functional code is easier to understand
- Refactoring C++ is painful due to lack of refactoring tools
Often build time increases too much due to C++ linking model - Refactoring tools that you can count on
Means you can refactor without having all tests in place beforehand - Language designers are focusing on the wrong things
Everything that’s successful is going to be legacy at some time- Missing language level support for dependency injection and other
testings, such as access to private methods and variables like
with the C++ friend keyword - Make testing a first-class language component
- Missing language level support for dependency injection and other
- Never hide a TUF with a TUC (problems with testability)
- TUF: Test unfriendly feature: File IO, DB access
- TUC: Test unfriendly construct: Static methods
- Understandability is the biggest problem in programming
- Good practice: Add behavior with meta-programming, but never remove.
The source should never lie (Liskov substitution principle),
especially for dynamic languages - Main problem with legacy codebase
- It takes too long to make code changes
- Only if architecture is wrong should entire system be re-written
Rewriting is more of a political decision
- Is legacy negative? Depends on perspective. Take a more positive view
- Code is organic: garden metaphor
1.3 Evolving the key/value programming model to a higher level
- Billy Newport, IBM distinguished engineer
IBM WebSphere extreem scale chief architect - Product specific talk
- Redis: Key/value store available at Google code
- Efficient
Only writes to disk on demand. Attempts to do as much as
possible in memory - Supports common operations
- Add/remove operations
- Higher level increment and set based operations
- Trim set
- Intersection/union/count
- Stack-based pull/pop (left/right)
- Takes KV programming up a notch by raising the abstraction
- KV data model scales well in extreme scale
- To see the entire entity, lots of lookup-operations are required
Good that Redis support 100k ops/second. Most production KV
stores, though, support batch operations - KV means schema free development
- Easy to extend “schema”, adding new column
- Value is just a blob
- Sadly, Redis lacks transaction support
- No concept of unit of work
- Oops, user’s password didn’t get written, so Billy can’t login
- Lacking transaction support makes debugging very hard
- KV model lacks type checking because everything comes back a blob
Add type-safe methods on top - How to partition sets of KVs into subsets?
- Helps alleviate loading a lot of data upfront and sending unnecessary
data across the network
- Helps alleviate loading a lot of data upfront and sending unnecessary
- Instead of iterating 4m items within a page on submit of Twitter post
Create a closure on client and send it to server and have server
do KV operations, so data doesn’t have to go across network - Originally Twitter used the optimistic approach
Now switched to the closure approach - Closure approach ~ stored procedure for relational databases
- Asynchronous closure
- Good for performance
- Relies on underlying queuing model
1.4 Techniques that still work no matter how hard we try to forget them
- Analysis
- With agile we reinvent understanding
Don’t go to dataflow as with structured programming, but use
FitNesse and BDD
- With agile we reinvent understanding
- Architecture
- Has become everything and nothing
- Stack (as in network stack and framework stack)
Has become an overused metaphor. A stack implies a top and
bottom, but systems are more like on the same level- Systems A and B and membrane (interface) between the two
- As you move down the “stack”
Things no longer get increasingly more technical
- Architecture has gotten a bad name due to architect astronauts.
They suddenly show up, display PowerPoints, and disappear - Every system has an architecture
- Modeling
- E.g., only model through Rational Rose. Don’t do that
- Come to be because software is through of as mathematical modeling
That’s not how engineers see it, though - Modeling is about getting answers
About creating a simplified view of the world, leaving out
unnecessary details - Modeling is NOT about adding everything in
Like when generating code of Rational Rose
- Why we aren’t moving forward as a profession
Half-life of a professional programmer causes the profession to
re-invent stuff every 7-14 years, when programmers get promoted
to jobs no longer involving code
1.5 Making use of patterns
- Martin Fowler
- Instead of reading whole books
Chunk up books like with Refactoring and PoEAAP - Patterns aren’t for beginners, but for fairly competent people
Patterns doesn’t make sense for a beginner - People tend to become “pattern happy”
They see patterns everywhere, too many for everyone else - Pick up any new technique, and applying it too much is part of
the natural learning process. Only when you overuse, you learn
the real boundary of the technique. - A pattern should consist of:
- Name (establish jargon)
- How
- When
- Examples
- Don’t go for learning software patterns in general
It’s like going for learning paragraphs. It doesn’t make
sense. You have to narrow it down and pick a subject
1.6 xUnit test patterns: Refactor test code to improve ROI
- Presenter wrote book on unit testing patterns
- Like Clean code, except for unit tests
- Patterns for clean test code =>
Not spending too much time maintaining brittle tests - Why write unit tests in the first place?
To write self-testing code that lets us better react to change - With 50/50 production/test code
How do you avoid software becoming 2x more expensive?
ROI comes from better maintainability of production code - How to keep costs of maintaining tests low?
- Identify test smells and patterns
- Code smells
- Behavior smalls: test behaves badly
- Project smells
- Smells may relate to each other/cause each other
- Identify test smells and patterns
- Avoid shared fixtures
2 Tuesday, Oct 6, 2009
2.1 Keynote: Classes Jim, but not as we know them
- Simon Payton-Jones
- langpop.com: Haskell not much used, but much talked about
- Type classes != OO classes
- How to generically compare any datatype?
- Square :: Num a -> a -> a
- Square only works on numbers and Num is the type class
- Similarly Eq type class on types that can be compared
- Type classes work by passing a dictionary of functions on a
type along with the function call. Compiler adds the extra
parameter to each function call - QuickCheck: Compiler generates test suite
- Whereas objects are made up of member variables and methods, in
Haskell functions live independent from member variables (method
suite detached from values). It’s the method suite that’s passed
as an extra parameters by the compiler- Type based dispatch vs. value based dispatch
- Passing dictionary is like passing a proxy for the class
- Haskell class ~ interface in C#
- In Haskell types can be Numerable, Showable at the same time
Existing types in Haskell can have the class added as an afterthought - Type inference vs. explicit declaration on binding variable
- Because of subtyping in OO languages, type inferencing is hard
- Subtype may override parent at runtime
- C# supports constraining generics
- Haskell doesn’t support subtyping
- Power of the dot envy in OO
2.2 Agile modeling and documentation “best” practices
- Scott Ambler, IBM Rational
- Agile modeling
- Processes and practices for how to model in agile
- Partial method like XP or Scrum
You need to piece things together to cover whole spectrum - Do just enough modeling to get job done: otherwise gold plating
- In real world you often move beyond good enough because you don’t
know when things are good enough (consumer != producer) - When doing too much modeling, you commit to ideas too early
Once you spend time modeling the idea (before starting to
code), you tend to stick with the idea though it’s not good - When you don’t have easy access to stakeholders, you tend to
model too much upfront - Typical agile projects spend 3-4 weeks on setting up project
Architecture, initial requirements, development environment
- Agile scaling model
- Start with core agile development (Scrum, XP)
- If doing TDD, you end up doing less modeling upfront
Because you think through your model more as you go - Beware: detailed requirement SPECIFICATION is really detailed
requirement SPECULATION - Store documentation only at one place, the best place. Single
source of information, e.g., instead of visual model and code,
generate visual model from code
- Agile documentation strategies
- Prefer executable documentation
- Working directly with actual audience. Don’t guess because
then you’ll write too much documentation - Documentation gets out of sync with code => don’t write all
that documentation. Maintenance programmers only use a few
diagrams, not all the other stuff - Leave documentation to a later stage until your source has
stabilized. Maybe write the documentation in the next
iteration, especially for short iterations of 1-2 weeks - Recognize that documentation is the least effective way of
communication- The CRUFT formula of a document
- C x R x U x F x T = effectiveness
- The CRUFT formula of a document
- Model != documents
- Too many equate the two, so we have bad experience with
models. Models may be simple white board sketches or index
cards
- Too many equate the two, so we have bad experience with
2.3 Actors – Towards OO modeling of concurrent systems
- Dave Thomas (Baderra) + Kresten Krab Thorup
- Based on 20-30 year-old research
- Performance/scalability have increased with multi-core CPUs
But our ability to understand in terms of parallelism hasn’t
increased that much. We need to abstract the concurrency away.- Explicit/reactive
Threads, Erlang/Corba/message middleware - Implicit/functional
Map/reduce, Google Hadoop, Data parallelism
- Explicit/reactive
- Problem with OO: OO is mostly ignorant with respect to concurrency
- Actors may provide an OO conceptual model for concurrency
Locking, mutex, etc. are really more techniques because we’re
lacking a conceptual model - Actor model
- Actor is a programming language structure
Client -> message -> queue -> actor - Actor encapsulates state regarding work to be done
Message is sent to actor asynchronously - Scala language supports actors
- Axum brings the actor language to C#
- Constructs a loosely coupled system of method calls
With each call being made asynchronously - Painpoint: Coordination is hard => debugging is hard
- Actor is a programming language structure
2.4 Achieving effectiveness with enterprise applications
- Michael T. Nygard
- Buy or build? Or download (open source)? Or software as a service?
- Goal: Select option that minimizes TCO
- But that’s a bad goal
- Seen cost vs. unseen cost (TCO doesn’t take these into account)
- Companies need one throat to choke, to hold responsible
- You can’t outsource accountability
- To be effective in the market, increase tempo
- Focus on adaptability
- Centralized adoption
Core group within organization decides on everything (IM
client, messaging framework) - Decentralized adoption brings more tempo
Decentralized adoption is hard to manage and it’s hard to
reach a shared goal - Take best of both worlds, aka. chaotic adaption
- Centralized adoption
- Focus on adaptability
- Commander’s intent
Military: Commander communicates intent TWO levels below. So,
2nd level has the same local freedom in achieving intent.
Understanding of purpose changes execution (fingerspitzengefühl) - Implementation may assume that people in the trenches don’t
understand big picture and vice versa. Caused by missing
communication loop - We developers look at automation as a good thing, when it’s
really technology neutral (everything manual shouldn’t be
automated) - Only automate to empower people, not to control them
- With, say, a new CRM system, you’re often better of changing
processes to reflect how software works rather than vice versa - Defining requirements up-front (through a workshop) on low
grounds of knowledge is bad - Custom build software
- Misplaced focus: The ones supplying requirements aren’t the
ones ending up using the system (identify major users early
on) - Drudgery: No one really depends on it, no one looses their job
if it goes bad - Every piece of software should have link to product blog,
discussion forum, and bug tracker in footer of every page
- Misplaced focus: The ones supplying requirements aren’t the
- Cathedral and the bazaar
Raymond is misquoted: Cathedral isn’t MS and everything else
open source. Raymond meant both to be open source - Thinking styles
- Reactive thinking (fire fighting) is a bad long term strategy
because it slows your long term tempo - Altitude induced arrogance: Because of higher level, higher
title, I know better
- Reactive thinking (fire fighting) is a bad long term strategy
- Service death spiral
As customer squeezes provider, provider takes out expensive
people from the project and puts more junior people on, lowering
the tempo, because junior people don’t know what they’re
doing. So customer gets more and more angry with provider and
after n years fires provider and goes with a new one and the
spiral starts over - Close feedback loops between all levels of organization
- Blog, forum, bug tracker
- Source of bad input must clean up after themselves
- Bridge gab between developer and user
=> decreased transaction cost
=> increased adaptability
2.5 Max value, max speed. How business and IT can collaborate to deliver
- Dave Thomas
- Support leaders that don’t know what they’re doing
Most leaders don’t - Bring the engineering team and the customers together
- Agile has no basis for being more productive
- Black$ (income), Red$ (expense), Yellow$ (waste, avoid)
Don’t go to Yellow$ meetings. Focus on Black$ and beware of Red$ - Lack of tangible requirements
If we don’t know how high the bar is, we don’t know if we’re over it - TDD should really be called example driven development
- Establish acceptance criteria early on. How to test the deliverable?
- In software, prototyping is a dirty word, but in hardware it’s
considered normal - Architecture is a role, NOT a job. Everyone should architect a
little to get dirty - 80% of IT apps are CRUD
Make them data-driven, not OO
Don’t take records and turn them into objects when the objects
don’t even have behavior. Then you stream objects in the largest
possible format and send XML across the wire
3 Wednesday, Oct 7, 2009
3.1 Keynote: C++, Java and .Net: Lessons learned from the Internet age
- Cameron Purdy/Oracle
- Why Java took over from C++
- Automated GC (Caller/callee cleaning up?)
- The build process (C++ building is slow)
- C++ compiler not only does compiling but linking
- Separation of code into .h and .cpp files
- Some things MUST be in .h file
- Simplicity of source code and artifacts: one class = one file
- Binary standard
- As opposed to one standard for Win32, Win64, Solaris
- C++ was about language, not binary standard. .h has to
be shipped for others to consume a library
- Dynamic linking
- Link on demand in Java
- No dynamic linking in C++
- COM is really about doing dynamic linking in C++
- Portability
- Java portable with little effort
- C++ isn’t portable because of missing standard libraries
- STL isn’t very encompassing
- Significant difference from compiler to compiler
GNU compiler is the one that adheres the most to
standards. With Java you only need a runtime on each
platform
- Standard type system
- An int isn’t an int isn’t an int (16, 32, 64 bit)
- There’s no bool in C++
- Generally, there’s a lack of portable types
- Libraries don’t work together when lacking primitive types
- Reflection
- Full RTTI in Java, enables generic frameworks
- Very useful for, say, ORM
- Performance
Multithreading in C++ is > 3x slower than in Java because of
thread safety in C++. In C++ everything has to be thread-safe
because C++ doesn’t have enough RTTI to determine if calls are
safe. The Java Runtime Environment can do various analyses at
runtime - Safety
- Elimination of null points (shows the arcane nature of C/C++)
- No buffer overruns in Java
- Bounds checking
- Fugly: Fucking ugly
C++ templates get expanded cut & paste at compile time rather
than at runtime
- C++ benefits
- Startup time
- Memory footprint
- No full GC-pauses (Java has to alloc everything)
- No barriers to native integration (OS’s are build in C/C++)
- With Java running on server, we no longer have to build GUIs
on the client. Also, with a server, startup time no longer
matters. That takes away the last benefits of using C++ - If you’re building an app for the cloud, you aren’t building
for the cloud. Building for the cloud should be done in a
transparent way
3.2 Guiding your personal life: Plan-driven or agile
- Industrial revolution => clocks for everyone
- Before clocks => slept till sun got up
- In factory life, clocking everything permeates into our
personal lives. To get up and work on the clock, we drink
caffeine. Caffeine passes easy across blood/brain barrier
(effective within 1h). When it’s time to sleep, caffeine
says: later - We minimize sleep to be more effective
- Sleep deprived brain shows signs of premature aging
- We learn as we sleep
- Caffeine no better than breaks
Only improves performance for simple tasks, research shows - Spiders on drugs (NASA study)
- Poincare
“I’ve a lot of problems to solve, so I’ll go to bed early” - Get of the treadmill
- Learn, focus, work hard, take a break
- Sounds agile
- Sleep phases: Shift from NREM -> REM at 90 min intervals
- Humans aren’t designed to be linear, but to pulse
Source: Harvard business review - 90 min. is a magic number – human cycling
- Pair programming maybe should also last 90 min
- Experiments are what you should do with agile, but we don’t
experiment enough with agile - Assume a “beginners mind”. Don’t just assume prior knowledge
- Start and cultivate new options
- www.pomodoro-book.com: Free book on the Pomodoro technique
About to come out as a real book - InfoQ.com: Search on Linda Rising and “Born to cycle”
3.3 ORM += 2: Making the most out of NHibernate
- Considering how important the db schema is for any application,
fixing the schema early on is stupid => instant legacy problem.
People work so because it’s so hard changing the schema - Use NHibernate to manage the schema itself
- Use SchemaUpdate to have NHibernate manage the database. Add a
new field in code, call SchemaUpdate, and the database is now in
sync with the code. So to get started doing development, you
don’t need server manager open, just VS and NHProfiler - SchemaExport and SchemaUpdate can do destructive updates to
database, but also just generate necessary SQL scripts - You can turn on validator and validate schema and code work
together. So you don’t need to navigate to a page to see a
database mismatch error
3.4 Deliberate practice in software development
- Mary Poppendieck
- What’s talent?
- Def 1: Nature
- Def 2: Nurture
- To become an expert is where deliberate practice comes in
According to science we’ve yet to proof that anybody’s mind is
limited from becoming an expert. Conclusion: It’s about
deliberate practice - 15 years of research has led to remarkably consistent findings
across a wide array of fields: Deliberate practice leads to
expertship - The 10 years rule: Deliberate practice leads to expertship
- What’s deliberate practice?
- Identify specific skill that needs improvement
- Device focused exercise to improved skill
- Practice
- Obtain immediate feedback (and adjust practice)
- Focus on pushing limits (expect repeated failures)
Because of constant failure, you need high motivation. You
need high motivation to keep at it - Practice regularly and intensely for 3-4 hours/day
- Four Elements of deliberate practice
- Mentor
- In the agile practices, where’s the mentor? He’s not there.
Most organizations don’t have a competency leader. No one to
teach others - Tasks of mentor involves
- Hire and grow people
- Review and guide work
- Teach and provide feedback and guidelines to others
- In the agile practices, where’s the mentor? He’s not there.
- Challenge (don’t always do easy stuff)
- If it’s hard, do it frequently and it’ll get easier
- In agile: Test/integrate/release (early/often)
- Assign work to challenge and develop people
- Don’t do what’s easy, look for challenges
- If you don’t sometimes fail, you’re not getting better
- Are developers kept at arm’s length from customers?
Do they lack immediate feedback?
- Feedback
- TDD, continuous integration
- Design reviews
- Common code ownership (others look at your code)
- Customer feedback in each iteration
- Open source projects provide immediate feedback
- Committer reviews
- Community feedback
- Dedication (keep doing it)
- Skill development takes time
- Time to learn: focus time without interruption
- Motivation
- Do a job you’ll love
- You don’t know why you’re doing it, you just are
- Develop a career path for technical experts
- Pride of workmanship
Know why you’re doing what you do
- Mentor
- Toyota improvement kata
- To gradually improve, mentor asks Socratic questions, i.e.,
questions that teach the mentee to learn. So no question like
“have you tried x?”. Mentors don’t provide answers - Pair programming is somewhat of a mentoring experience
Knowledge transfer goes both ways
3.5 Umbraco intro @ Jaoo 2009
- Niels Hartvig
- Started as a prototype in 2001
- Open source, web CMS
- Umbraco is different from other CMSs: community-driven
- 4,500 active community members
- Respects craftsmanship
- Requires knowledge of how to use basic tools (asp.net, IIS)
- Intension is to handle the boring stuff, yet make regular
development possible - 100% control of markup
- With Umbraco everything is a hierarchy
- Data isn’t relational
- Runtime data is XML
- Honors work of W3C: makes use of XML, XSL
- Everything is a list, everything is an element in a list
- Only three concepts to learn compared to regular development
- Document types
- Templates
- Macros
- You can use VS instead of editing source files in the browser
by importing all VS-relevant files from your Umbraco project
3.6 Value management with Scrum development
- Kai Gilb
- We have a challenge
Management doesn’t get agile. Agile TODAY is very much from the
developers point of view. It doesn’t deliver value to
stakeholders within agreeable resources - Agile manifest doesn’t talk about external value delivery and
stakeholders. It’s all centered around developers - Likewise, Scrum isn’t focused on value delivery and
stakeholders and what they value - Before looking at technologies, focus on management and value.
The delivery to these people should be increased value, usually
in the form of code - Before each Scrum iteration, gather measurements from management
- Stakeholders shouldn’t push technical solutions, but value.
The Scrum team should be the ones to come up with technical
solutions, e.g., when stakeholders say “password”, what they
mean is security - As a developer, if you can deliver value, you’re less likely
to get outsourced - Software developers don’t deliver software, they deliver value
to stakeholders. The software is only our tool
