Document Actions

Cosent Blog

Co·sent: shared perception; joint knowledge; collective intelligence.

Plone Intranet München sprint report

Guido Stevens | Mar 14, 2015 | permalink | open source · plone

Sprinting in München transformed both the team and the code base of Plone Intranet.

Plone Intranet München sprint report

The Plone Intranet project represents a major investment by the companies that together form the Plone Intranet Consortium. Last week we gathered in München and worked really hard to push the Mercury milestone we're working on close to an initial release.

Mercury is a complex project, challenging participants out of their comfort zones in multiple ways:

  • Developers from 6 different countries are collaborating remotely, across language barriers and time zones.
  • People are collaborating not within their "own" home team but across company boundaries, with people whom they haven't really collaborated with before, and who have not only a different cultural background but also a different "company coding culture" background.
  • The backend architecture is unlike any stack peope are used to work with. Instead of "normal" content types, you're dealing with async and BTrees.
  • The frontend architecture represents a paradigm shift as well, requiring a significant change in developer attitude and practices. Many developers are used to work from the backend forward; we are turning that on it's head. The design is leading and the development flow is from frontend to backend.

So we have a fragmented team tackling a highly challenging project. The main goal we chose for the sprint therefore, was not to only produce code but more importantly to improve team dynamics and increase development velocity.

Monday we started with getting everybody's development environments updated. Also, Cornelis provided a walkthrough of how our Patternslib-based frontend works. Tuesday the marketing team worked hard on positioning and communications, while the developer teams focused on finishing open work from the previous sprint. As planned, we used the opportunity to practice the Scrum process in the full team to maximize the learning payoff for the way we collaborate. Wednesday we continued with Scrum-driven development. Wednesday afternoon, after the day's retrospective, we had a team-wide disussion resulting in a big decision: to merge all of our packages into a single repository and egg.

The big merge

Mercury consists of 20 different source code packages, each of which had their own version control and their own build/test tooling. This has some very painful downsides:

  • As a developer you need to build 20 separate testing environments. That's a lot of infrastructure work, not to mention a fiendishly Jenkins setup.
  • When working on a feature, you're either using a different environment than the tests are run in, or you're using the test environment but are then unable to see the integrated frontend results of your work.
  • Most user stories need code changes across multiple packages, resulting in multiple pull requests that each depend on the other. Impossible to not break your continuous integration testing that way.
  • We had no single environment where you could run every test in every package at once.

So we had a fragmented code base which imposed a lot of infrastructure work overhead, created a lot of confusion and cognitive overhead, actively discouraged adequate testing, and actively encouraged counterproductive "backend-up" developer practices instead of fostering a frontend-focused integrative effort.

Of course throwing everything into a big bucket has it's downsides as well, which is why we discussed this for quite some time before taking our decision.

Code re-use

The main consideration is code re-use and open source community dynamics. Everybody loves to have well-defined, loosely coupled packages that they can mix and match for their own projects. Creating a single "big black box" ploneintranet product would appear to be a big step backward for code re-use.

However, the reality we're facing is that the idea of loosely coupled components is not how the code actually behaves. Sure, our backend is loosely coupled. But the frontend is a single highly integrated layer. We're building an integrated web application, not a set of standalone plugins.

We've maintained the componentized approach as long as we could, and it has cost us. A good example is plonesocial: different packages with well-defined loosely coupled backend storages. But most of our work is in the frontend and requires you to switch between at least 3 packages to make a single frontend change.

In addition, these packages are not really pluggable anymore in the way Plone devs are used to. You need the ploneintranet frontend, you need the ploneintranet application, to be able to deliver on any of it's parts. Keeping something like plonesocial.activitystream availabe as a separately installable Plone plugin is actively harmful in that it sets wrong expectations. It's not independently re-usable as is, so it should not be advertised as such.

We see different strategies Plone integrators can use ploneintranet:

  1. Light customization.

    You take the full ploneintranet application and do some cosmetic overrides, like changing the logo and colours of the visual skin.

  2. Full customization.

    You design and develop a new application. This starts with a new or heavily customized frontend prototype, which you then also implement the backend for. Technically you either fork and tweak ploneintranet, or you completely build your own application from scratch, re-using the ploneintranet parts you want to keep in the backend via library mode re-use, see below.

  3. Library mode cherry-picking.

    You have a different use case but would like to be able to leverage parts of the ploneintranet backend for heavy lifting. Your application has a python dependency on those parts of ploneintranet you want to re-use: via ZCML and GenericSetup you only load the cherries you want to pick.

Please keep in mind, that this situation is exactly the same for the companies who are building ploneintranet. We have those same 3 options. In addition there's a fourth option:

  1. Extension.

    Your client needs features which are not currently in ploneintranet but are actually generally useful good ideas. You hire the ploneintranet designer to design these extensions, and work with the ploneintranet consortium to develop the new features into the backend. You donate this whole effort to ploneintranet; in return you get reduced maintenance cost and the opportunity to re-use the ploneintranet application as a whole without having to do a full customization.

You'll have to join the Plone Intranet Consortium in order to pursue this fourth strategy. But again, there's no difference for current members: we had to join as well.

To make individual component re-use possible, we've maintained the package separation we already had - ploneintranet may be one repository, one egg, but it contains as separate python packages the various functional components: workspace, microblog, document preview, etc. So we do not subscribe to JBOC: Just a Bunch of Code. We don't throw everything into one big bucket but are actively investing in maintaining sane functional packages.

A variant of cherry-picking is, to factor out generically re-usable functionality into a standalone collective product. This will generally only be viable for backend-only, or at least frontend-light functionality, for the reasons discussed above. A good example is collective.workspace: the ploneintranet.workspace implementation is not a fork but an extension of collective.workspace. This connection enables us to implement all ploneintranet specific functionality in ploneintranet.workspace, but factor all general improvements out to the collective. That has already been done and resulted in experimental.securityindexing.

Current status

On Thursday we announced a feature freeze on the whole stack, worked hard to get all tests to green and then JC performed the merge of all ploneintranet.* into the new ploneintranet consolidated repo. Meanwhile Guido prepared the rename of plonesocial.* to ploneintranet.*. On Friday we merged plonesocial into ploneintranet and spent the rest of the day in hunting down all test regressions introduced by the merges. Because we now have a single test runner across all packages that meant we also identified and had to fix a number of test isolation problems we hadn't seen before.

Friday 20:45 all tests were finally green on Jenkins!

We still have to update the documentation to reflect the new consolidated situation.


In terms of team building this sprint has been phenomenal. We've been sprinting on ploneintranet for five months now, but this was the first time we were physically co-located and that's really a completely different experience. We already did a lot of pair programming remotely, but it's better if you are sitting next to each other and are actually looking at the same screen. Moreover feeling the vibe in the room is something you cannot replicate remotely. The explosion of energy and excited talking after we decided to do the consolidation merge was awesome.

On top of that we now have a consolidated build, and I can already feel in my own development the ease of mind from knowing that the fully integrated development environment I'm working in is identical to what all my team members are using, and is what Jenkins is testing. Instead of hunting for branches I can see all ongoing work across the whole project by simply listing the ploneintranet branches. Reviewing or rebasing branches is going to be so much more easier.

On top of all that we also made significant progress on difficult features like the document previewing and complex AJAX injections in the social stream.

We started with a fragmented team, working on a fragmented code base. We now have a cohesive team, working on a unified code base. I look forward to demoing Mercury in Sorrento in a few weeks.

Permalink »

Death of Sharepoint triggers search for intranet alternatives

Guido Stevens | Mar 03, 2015 | permalink | intranet · technology

IntraTeam conference highlights ongoing disruption in intranet market.

Never mention the word "intranet" on a date, or in any conversation for that point. It bores people to death.

Ask a whole auditorium of intranet managers: what emotions does your intranet evoke? Beyond indifference, the answers you get are: resignation, frustration, rage, desperation, contempt.

Intranets are dying

Throw an intellectual heavy weight like Dave Snowden into that mix, and he'll happily challenge some of the audience's dearly held beliefs.

"The future is distributed. I don't believe, in five year's time, there'll be significant presence at any conference to do with intranets."

"The intranet is going to die. We're moving to fully distributed systems. The sooner you start shifting the better."

Snowden thinks, apps are much better at playing this new field of distributed information and knowledge management.

Sharepoint is dying

Microsoft appears to have arrived at the same conclusion as Snowden.

"Office 365 is a collection of small applications, maybe loosely tied together. Or completely siloed."

And this app based system is what replaces SharePoint, up to now the dominant software package for building intranets.

"The next version of SharePoint is Office 365, and it is already here. SharePoint 2015 will just be a fancy service package."

Perttu Tolvanen, an analyst at North Patrol, makes the point that Microsoft's big money maker is Office, and the whole company strategy is aimed at protecting that asset. SharePoint used to support Office, but now that Office has moved to the cloud in the form of Office 365, on-premise SharePoint is no longer needed.

"For Microsoft, it's a supporting business for Office. That's why Microsoft has decided to let go of this business. SharePoint 2013 is a dying system. ... All the product development focus of Microsoft is in Office 365."

Other analysts at the conference confirmed this conclusion. It's a public secret anyway.

Let me put that into context. On an intranet conference, you meet someone with the job title "digital workplace application analyst" - that's a SharePoint application manager if you talk to them. Or you encounter a "technology independent intranet consultant" - you'll find out that their bread and butter is as a SharePoint project manager. It's as if SharePoint and intranet were close synonyms.

SharePoint is a swiss army knife system strong enough that IBM and Oracle had given up on the intranet and document management market.

And now the 800-pound gorilla dominating the industry just -poof- disappears!

WCM best positioned to take advantage

On first sight, the Office 365 cloud offering looks similar to the SharePoint on-premise version. But if you look closer, you'll find that complex portals or integrated digital workplaces cannot be migrated to the cloud. You need a complete re-design to bring your existing intranet to Office 365.

"Microsoft doesn't care for the on-premise customers for the next five years. There will be a lot of customers looking for alternatives to SharePoint, once they realize that SharePoint 2013 is dying."

Web Content Management systems and portal systems are already better than SharePoint for building complex systems with multi-language capabilities and integrations with other systems. The departure of SharePoint from the intranet market will accelerate the search for alternative solutions. The departure of the dominant supplier leaves a lot of extra oxygen available for the rest.

Return of the portal

What we're seeing here, is a transition from the outdated intranet concept, to a new digital workplace paradigm.

Dave Snowden:

"We're moving into a radical new approach to software which is fully distributed. The only interesting question at the moment is: what is the glue that holds it all together? That is probably the big strategic area to grasp."

Paradoxically, one of the contenders for the integrative part that brings everything together is a revival of the portal concept.

The screenshot below is from James Robertson's presentation and shows how a HR page becomes dramatically more useful by pulling relevant, personalized information from various back-office systems.

screenshot of Telstra HR portal

The blurring is a design feature by the way. There's a button to un-blur your pay info when nobody is watching over your shoulder.

Seeing that screenshot, Kristian Norling tweeted:

Thinking that ... the portal strikes back #IEC15

And that's a something that Perttu Tolvanen also referred to.

This is not your grandfather's portal anymore, though. Snowden again:

"The other point is, things will be loosely coupled. This, by the way, is where object orientation comes in big time, but true object orientation."

That's a vision similar to what Prahalad and Krishnan in their 2008 book The New Age of Innovation.

Connecting heterogeneous networks of loosely coupled business objects is core business for web technology. Open standards, open source approaches are especially well placed, to thrive in such environments.

Permalink »

3 signs your communications project is off-track, and how to fix it with personas

Well-researched audience personas communicate deep insights

3 signs your communications project is off-track, and how to fix it with personas

The meeting was a disaster. It was supposed to be a "rubber-stamp" type of event, just to discuss a few questions, after the proposal and the budget had been approved already. After the design had been approved half a year ago, already.

Unfortunately, some key questions could not be answered.

3 questions to check your communications vision

To validate any communications project, three basic questions are useful:

  • Who are we serving with this project?
  • What will be the difference for them, a year from now, if we succeed?
  • How will we interact with our audience and improve their lives?

Boom. The client has some ideas, just enough to invalidate the current design. But at the same time, those ideas are vague enough that it is not possible to articulate design directions, or make decisions.

The standard Dutch reflex to such situations is: we need to have some more meetings to discuss this (codespeak for: let's try and negotiate our differences away).

That reflex is wrong. You don't need to negotiate opinions. You need more data.

Personas synthesize research based insights

If you're building web sites, or any service that people interact with, you need a clear picture of who your intended audience is. You don't get that picture by sitting at your desk. You need to get out of the building and interview real, live humans to find out what makes them smile. To find out how exactly your service can add value to their lives.

Once you've done that research, you can then summarize and synthesize the findings into audience personas, profiles of fictional people that represent a typical customer. Persona's show describe key aspects of a person's life, goals and behaviors. Below are some example personas, based on Mailchimp user research:

example personas developed by Mailchimp

Those personas then drive your design. They enable you to empathize with your audience. They enable you to create a design that touches people in their hearts. And they enable you to overcome the sometimes weird ideas of people holding purses, who happen to like the color "red", by grounding design decisions in a solid, data-driven understanding of audience needs and preferences.

Buyer personas

Originating in the realm of web design and service design, the use of audience personas is now also gaining traction in the world of marketing, in the form of buyer personas.

Instead of making stuff up about your customers, you can ask them for the five key insights you need to focus your strategy:

  1. Priority initiatives

    What business need drives them to search for a solution in your market space?

  2. Success factors

    What do buyers expect to achieve by implementing your solution?

  3. Perceived barriers

    Which were the reasons to not buy your solution? Make sure to also interview non-customers!

  4. Buyer's journey

    Who is involved in the decision making process, what resources are trusted?

  5. Decision criteria

    Which factors are key in weighing alternative options and making a purchasing decision?

Gaining a deep understanding of these five points is key not only for buyer personas, but for design personas in general.

If you've articulated these five insights, you'll know the answers to the who, what and how questions. You'll know who you're serving, what value you're adding, and how this fits into the lives of your audience. The rest is execution.

The takeaway

  • Don't be fuzzy.

    If you don't know the who, what and how: acknowledge you don't know enough.

  • You don't need to guess.

    Persona creation is a proven methodology for crafting data-informed audience profiles.

  • Just do it.

    Doing the legwork to develop true insights is hard work. But it's actionable, not magic.

  • Use it or lose it.

    Services and businesses that invest in a solid evidence-based understanding of their audiences' needs, will out-shine those who design and market is based on hunches.

Permalink »

Plone Intranet Consortium

Guido Stevens | Dec 04, 2014 | permalink | intranet · open source · plone

Creating a new open source digital workplace platform.

At the Plone Conference 2014 in Bristol, Guido Stevens presented the Plone Intranet Consortium to an audience of Plone core developers. The presentation focused on the need for a new business model for open source development, and the specific traction we're achieving within the Consortium already.

Plone Conference 2014 - The Plone Intranet Consortium

The "spare time" model of doing open source is broken. We need a better model. In the Plone Intranet Consortium, a dozen Plone companies are jointly investing in the creation of a high-quality out-of-the-box open source intranet platform based on Plone. What is different is not just the funding model, but also the design-first approach we're taking.

Slides of the presentation are available on SlideShare:

Project Mercury

social intranet screenshot

The Consortium is now working on creating the alpha release, codename: Mercury, which is scheduled early 2015. We'll inform a wider audience via a website and a series of workshops, in the run-up to the Mercury release. Stay tuned!

Permalink »

Social intranet sprint Berlin

Guido Stevens | Sep 15, 2014 | permalink | intranet · plonesocial · plone

A week of intense collaboration in Berlin has significantly accelerated the development of a new Plone-based social intranet platform.

Social intranet sprint Berlin

The Plone social intranet sprint in Berlin at the Humboldt University was attended to by about 16 participants across two tracks, plus a social program in the evenings. Apart from all the hard work we also had a lot of fun. Berlin is a vibrant city and the Ploners who are also Berliners took us to some great restaurants.

strategy track

The strategy track saw enthousiastic responses to a plan for joint investment by ~10 Plone companies into a new Plone intranet software suite. We'll use the coming weeks to solidify the momentum for this initiative and try and convert positive intentions into hard investment commitments.

A key part of the proposed plan is a design-first process to create a compelling user experience, leveraging Plone5-compatible frontend technologies. Netsight, Cosent and Syslab have already made an exploratory first iteration with this design process, focussed on re-designing the microblogging and social activity stream interactions in Plonesocial.

coding track

In the coding track, we've taken these new frontend designs and implemented them on top of the existing Plonesocial code base. The result is, that we now have a Patternslib based Plonesocial implementation that can be installed and run in a Plone 4 installation. Because Plone5 Mockup is also Patternslib-based, the work we're doing is forward compatible and will be easily portable to Plone 5.

In addition to implementing the existing Plonesocial features, the sprint also resulted in the integration of plonesocial.messaging (private one-on-one messages) and new "reply" functionality (conversation threading).

The new frontend introduces a host of new features that are not yet provided by the backend and need to be architected and coded: file uploads, URL and file previews, "@mentions", "liking", "favoriting" etc. Also, we're already working on extending the design with a number of subtle but powerful micro-interactions in the form of shortcodes to provide a pluggable linking system.

The current development version of Plonesocial is more advanced than the last released version, but it needs more work before we can make a production-quality release - so come and join us in the next sprint at Plone Conference 2014 in Bristol.

Permalink »