Social Listening for Product Managers #SVPCAMP

Social Listening for Product Managers

Sudha Jamthe – @sujamthe
PayPal/Ebay
Stephanie Shum – @stephshum
Yahoo
Van Riper – @vanriper
Google

Why social is a channel for product manager?
Web 2 got customers voice online, Meetups brought them together locally, Social echos feedback

Grassroots movement of tech customers helping each other
Meetups and User Groups
DevCamps/BarCamps
StartupWeekends

For non-tech
Clubs
Interest groups
FB profiles

What is social listening?
1. Listen to your existing base
2. Listen on social channels
3. Global product listening

Listening to your existing base
Forums, Suggestion board, Webinars/chats, yahoo groups, google groups, linked in groups

Yahoo Group for Y! Mail
270 Million users 5-7 PMs on mail
Way to search for bugs or provide a response mechanism

LinkedIn for Mobile Group
PM is a community manager to engage conversation
Yahoo Suggestions (suggestions.yahoo.com)
Internal tool for customer feedback
Involve QA, Design, Engineer

Q: How do you categories and manage incoming questions?
A: Use Overtone to use text analysis. 

Q: How much time do you devote to this?
A: 1/2 day once a week. Bug bashes once a week.

Q: Do you have levels or different ways to collect feedback to respond to people?
A: Overtone, Customer Care has some tools. The forums work to help each other out.

Q: Is Yahoo! Classic Mail is going to be going away?
A: It’s classic. Will be supporting classic, or something similar for some time to come.

Q: How do you get a base of users if you don’t have a base?
A: Groups solutions are a way to get people together. Monitoring twitter, Facebook etc. Suggestions or even Overtone are for larger scale solutions.

Q: What happens next in the PM process?
A: Look at what the hot issues are and try to address those needs. Some times it requires further explanation and research to understand the core issues and problems.

Q: Who communicates back to users?
A: The whole team can and does respond. Specific bugs we deal with directly.

Q: Have you had a situation with nasty backlash? How do you deal with that?
A: We try to get them to get to the core issue. 
VR: Make sure you have guidelines. But these people are vocal and can be big supporters.

Listen on social channels
Social media for market analysis of features – See what people are saying about products in social channels
You can setup social listening for specific channels
PayPal includes transparency into the roadmap
Filter and get to one product at a time to get it to the right people
Streamlining social listening
– Tweetbytes: twitter chat using hashtag #tweetbytes
– PM and customer chat
– Metrics: # of feedbacks, # of new engaged customers, # of influencers engaged
Testing across multiple products @paypalx and @ebaydev

Q: Do you have an opinion on dashboards vs. individual monitoring?
A: Alterian SM2 and Radian6 are similar in dashboard views. The PM’s are given training to PMs to engage in social. Also, HootSuite or CoTweet to assign out to different people.

Q: Are you using social as a beta for new products?
A: Only on existing products. It could be but not right now.

Q: Do you use social for enterprise?
A: It should be effective, but you need to baseline to understand if they are in the channel.

Q: If Google is engineering focused and Yahoo is product driven, what is eBay?
A: Focused on eCommerce so customers make money.

Q: What is eBay’s perspective on social learning?
A: Probably a bit early for us. We are synthesizing the data.

Global product listening using user groups
Have a group to allow consumers of the Google product API’s to connect
These groups also provide data back into Google

Q: How do you interact with these people?
A: The groups are independent. There is a process to apply to be a group. There are admins and organizers for the groups. 

Q: What are common topics for the communications you send?
A: It is two way but it isn’t really marketing. Act more as an advocate.

Product Management is Really Innovation Management #SVPCAMP

Product Management is Really Innovation Management

Tom Grant
Forrester

Product managers own the bridge between the business and IT, improving the odds of successful innovation

What is innovation?
– Something new or new way of doing something
– Different
– Better
– Solution to a problem
– Makes doing task fun
– Innovating is using existing things in new way
– There is no right answer and is not clear

Innovation creates value
– Cloud
– IT to BT shift
– Device independence
– Agile/Lean
– Customer experience
– Social Media

Customers want solutions, not inventions
Customers want assurance the adoption will succeed, not hear about how impressive the technology is

Innovation is about mutual value
Challenge – who, what, why, when, how
Examples – We don’t understand our customers or what they value.
We have lots of ideas, but we don’t know how to choose among them.

Innovation is not just invention
FBI Case Study
2001 start project
2005 $100 million spent and nothing 
2006 $170 million and broken
2008 Still not live
2010 Still going

But invention is a key element of success
Moving to an Agile process
Team and stakeholders was not good
Requirement problem needs to be redefined

Even the best inventors don’t always innovate
Focus on invention, not adoption
HP Touchscreen
MS Kin phone 
Apple Cube
Google Wave

Academic studies of innovation also have gaps
Everett Rogers theory

Combining invention and adoption
The Innovation Process
Conceive -> Test -> Build -> Deliver -> Adopt -> Assess
Who owns this process from start to finish?
Most companies, it’s nobody

What is Product?
– Something people buy to meet a need
– Solution to a large problem
– Set of features/devices packaged as a solution
There isn’t a clear answer

This is a product, not a service
Wells Fargo is not too different from a SaaS vendor
Chevy Volt has 10 million lines of code, 100 electric controllers, with it’s own IP address
Services are trying to productize what they do and market each service

A simple definition of what product managers manage
A discrete piece of technology designed to make a valuable contribution to one or more large solutions.

Product define a quantum of value
– Compartmentalization
– Simplification
– Design
– Re-usability
– Maintenance
– Performance

Q: Have you seen a frankenstein project and have to make a leap to the next generation?
A: It is in the roadmap. It may not be something that could be easily maintained. 

What is product management’s role in innovation?
Conceive -> Test -> Build -> Deliver -> Adopt -> Assess (strategic job function)
PM’s have to do all of these things equally
– Responsible for the success of a product over the entire life cycle
– Manages product within the larger portfolio
– Collects, analyzes, and prioritizes requirements
– Translates between languages of engineers and business users
– Supports teams responsible for the delivery of the technology
– Rewarded based on adoption of technology

Product Managers shepherd products through innovation
Q: How are executives going to be managed?
A: It will take some risk in the organization. Many execs understand there are opportunities and can be open to it.

And that’s how top-notch PM’s see it
What is the minimum viable product? ->
Case validation -> Make an initial estimate of market potential
Product validation -> Use a particular vertical as a test case
Sales and support validation -> 
Scale-up validation 

“Real World” Strategies for Prioritizing Requirements #SVPCAMP

“Real World” Strategies for Prioritizing Requirements and Product Backlogs

Greg Cohen
280 Group LLC

Overview
Techniques for assessing value
Mitigating Risk
Mitigating Costs
Impact of time on priority

“Prioritize Requirements by Business Value”
Makes sense and we want things to create value
But doing it is harder than identifying the issues

Business Value
Improve the product or service for the customer (in alignment with business strategy)
Increase/protect revenue
Decrease/prevent cost
Maintain compliance

Must also balance risk/cost
Increase value over time

Challenges with prioritizing
Determine business value
Value is often realized by a collection of features (one bit is harder to show value)
Value determination usually requires judgement
Much of the wisdom is locked in individual’s heads (biases but isn’t articulated)

Techniques to Assess Value
1. ROI – to your business and your customer 
Q: Sometimes the measure is hard to measure, then what?
A: You may need to use another tool.
2. Customer Research
In depth interviews, focus groups, observation, innovation games, usability studies, support cases, usage and click stream data, surveys
3. Market Opportunity Scores
What is important? (what are users trying to accomplish
How satisfied are they? (is it enjoyable)
4. Kano Model
How does a solution not meet/meet a need?
How well does a solution leave the user dissatisfied/satisfied? 
Creates curves to show opportunities – Must haves, Performance, Delighters, 
Time pushes scores down
Q: How do you pick what goes into the releases?
A: Plot the features out on the model to understand what goes where. 

Q: What do you do if your business is so far behind the others?
A: There is a fast follow model. You need to find a way to be a differentiator.

Q: How would you influence customer must-have to transition to a delighter?
A: Difficult to do, perhaps through marketing. But the model isn’t designed to do that.

5. Buying Features
Buy a feature – Create list of features, play with 4-7 customers at a time, Price some features so customers have to pool money
Q: Do you look at individual users?
A: This is group methodology. 
Spend $100 of R&D – Create list of features, ask customer to allocate R&D budget to list of features

6. Subjective Value Determination
Set a vision for the product and place each requirement in the context of the vision
– Get stakeholder participation to unlock collective knowledge
– Prioritization Matrix is good for weighting priorities across existing and new customers

Risk
Mitigating Risk
Types – Technical, market, regulatory, delay
Weigh – What can be tolerated on a project, organization, etc.
Practice risk-first development – give priority to requirements that have risk

Let the team work with technical pay-off
– early learning and maximize understanding

Evolution of features
Identify phase of lifecycle for feature area
– POC
– Validate the learning
– Minimal marketable feature – what is the minimun required to work
Apply Kano model
– must have features to work
– Performance (enjoyment)
– Delighters (wow customers)

Cost
Moving risk to the front
Collaboration
Splitting
MMF
Alternatives

Time
High risk, new regulation, maintenance upgrades, microprocessor 
Risk, Return Cost View
Low Risk <–> High Risk
Low Return <–> High Return (vertical axis)

Summary
Prioritization is complicated
Balance return, cost and risk
Mitigate project risk

Q: How do you sell priorities to stakeholders?
A: Hopefully rational explanation will work. Can be hard.

Mobile Prototypes for Product Managers #SVPCAMP

Mobile Prototypes for Product Managers

Etay Gafni – 
Demo Right

Goals – Educating your teams/boss
– Why
– Who
– How

You’re going mobile
Big Market, Money and Buzz
But, Big risks, investments in new technology
Are you looking ahead or trying save/leverage past investments?

Keep in mind
Keep It Simple Stupid

Story – Writing in Space
Task: Design and develop a writing device
US: Spent millions for a pen
USSR: Used a pencil

Questions you should ask
Which operating systems?
Which devices?
Native or HTML5?
*Are you developing the right app? (for your customers)

Traditional Process
Is there demand for this mobile app?
Did we prioritize and validate features with our customers?
How can we create detailed specs for our distributed development team?
How soon can we monetize it?
Will my competitor see my application before my customers?

The Mantra
Fake it until you make it

Test it
Get constant feedback from customers
Build the right app with clear specs
shorter time-to market
longer ‘shelf-life’ for the app

The dark side of prototyping
The prototype is the product
The prototype is NOT the product
Everyone is a designer/developer/product manager

Process
Create -> Test on target audience -> Collect Feedback -> Analyze -> Improve
After Analyze -> Lock -> Spec -> Demo/Dev(branch)
Question: How does it differ from web prototyping?
A: Not really different, just keep in mind it is mobile.
Q: How long to you schedule?
A: Depends on scope. You have to ask the right questions. 

The main steps
Create, Share, Analyze

Fidelity Levels
Wireframes – basic interaction (paper, ppt)
VP Frame – sketch for Vice President
High Fidelity – designer based

Tool Kit: Paper
The good – fast, cheap, mobile 1/1 feedback
The bad – just low fidelity, hard to manage versions, hard to scale
The ugly – as good as your handwriting

PPT to Photoshop
The good: Fast, Cheap, You know how to use, audience can view
The bad: Focus on high fidelity, not mobile, hard to manage versions
The ugly: Everyone is a designer, can’t collect feedback

Code
The good – real code, can publish
The bad – expensive, technical, is it right?
The ugly – too late, ready to sell

All the templates are here: http://demoright.com/static/pcamp2011

Q: For small companies with a big client, what kind of tools do you use?
A: PDF’s work sometimes, but difficult for online feedback.

Q: How do you collect feedback?
A: Focus groups can be good but hard for mobile. Distributed focus groups can be a solution.

Q: Is it qualitative?
A: I want to understand how they use it without forcing them to tell what they did.

Q: Do you prototype workflow?
A: Yes. You want to know that the functions work. Transitions can come later.

Sharing/Testing
Select people that will use the product
Control the distribution
Test on device and online
Set expectations
Define tasks
Define success
Don not try to fix in real-time

Analyzing
Parameters  What can we learn
Clicks distribution  Navigation
Target Screens  Task completion ratio
Navigation flow  How does the user understand/use the application

Educating Your Boss
Why? Prototyping lowers the risks, You can lower costs
Who? Everyone can do it, no technical resource required, better if PM/Designers get a head start
How? Keep it simple, use tools you know, test on devices, share with your audience, ask the right questions

Q: How do you prioritize features?
A: You have to evaluate the feedback. Right audience, features, uses, etc. 

Q: What are good apps?
A: Angry Birds, Amazon

Opening Session “Go out and make something happen!” #SVPCAMP

Product Camp – Opening Session

Dane Glasgow – VP Product Management
eBay

15 years old
Auctionweb

Technology and Retail is evolving
Innovate faster, smarter to allow employees to impact the bottom line
Mobile, local, digital, social 
2010 $62 billion in sales, $2 billion through mobile apps 
2011 plan to double mobile
Milo allows cross-channel which is 5X the size of eCommerce

How we think
Customer at the center
Fashion, Tech, Tickets, Vehicles
Verticals have different requirements

Product Management
Last two years push to technical depth
3,000 people working on the site
Curriculum to emphasize training

Green Box – came out of Skunkworks to recycle boxes