I had never experienced the term Ghetto Testing until I read a blog post about how the FarmVille creators use it. One of the biggest parts of Ghetto Testing is to track interest and support for a new feature before actually building it. This means that before a single line of code is written, they throw up a link within the game that allows for people to sign up to be a part of that feature as soon as it become available. This is their way of testing interest. If enough people click on the feature, they will actually build it. If it is something that most people could care less about, they will go on to their next idea.
This seems to be entirely different than everything I currently do. Essentially, I create learning objects before anyone has said that they want them. I create courses that people have said that they want but that they are not intimately involved in developing. I produce blog posts that do not have a specific audience, and there is surely no way that I have of asking others for which direction I should be going in. Certainly, I get feedback in comments, but that is only after I have written out my first version.
So, this idea of Ghetto testing has really got me thinking about just how few iterations we really get as teachers or as workers. As teachers, at most we get to teach a single topic 4 distinct times a year (within a given unit of study), and most likely, we probably only get to teach it once or twice. The ability for a single lesson to be tested and iterated upon comes around so rarely that we are likely to either simply do what we did last time with a small adjustment or completely start from scratch.
But, what if students were able to gauge interest, and better yet, value in each discipline as they went through the curriculum. What if we could do a heat map test on which topics have the most interest from our students. What if we could build those items out only after we knew that it was something that they would use. Especially in terms of the way that they would like to learn a given topic, if we were able to present the materials in 10 different ways and we gauge the ways in which the majority would like to see it presented. We wouldn’t have to build all 10 ways, but probably just the top few. Then we could do some A-B testing to see which one was truly more effective.
Yet, we don’t do that because we have no mechanism for iteration. We only do A-B testing if we are forced to do action research. If there were some way of doing this on a large scale, some way to receive instant feedback on how we should be creating the curriculum, we could actually differentiate in the ways we say we should. Perhaps this is why I believe so much in hybrid programs. If we can allow students to choose their own adventure and then let them support those features that we haven’t built yet by simply being our beta testers, there would be so much intense buy in for doing well and actually making educational choices that would impact their learning.
And what about business… What if it were possible to do Ghetto testing with projects that you were working on. What if we gathered the early adopters for every new initiative in a company simply by engaging them in the process of self-selection. If CEO’s have the captive eyes of their employees, what would happen if they didn’t build the agendas for meetings but rather gathered the input from their interest in certain topics. It could change the ways in which people build new products, and the ways in which they create corporate culture.
Now, the blog post that started this line of thinking made sure to point out that you can’t always have the same people being the testers and you should try to test out too many unbuilt pieces at once. But, I don’t think that it would be a problem to release pieces of Ghetto testing within our own environments.
The question I am now faced with is “What do I want to NOT build today”? What should I put in front of people and let them make a decision based upon their interest? While the wisdom of a crowd is not absolute, creating something new (learning or a product) requires us to always analyze the data about the best way to introduce something that will (at least in the long run) be beneficial to them. Perhaps game developers have a point here… Boring is not an option, and people are interested in being a part of the learning/development process.
Hi Ben
Never heard of 'Ghetto Testing' either till now! I like the concept. No reason why it couldn't have a valuable place in education.
Patricia
I'm glad that you found this post useful. I think that there is a lot that
the Lean Startup community has to teach us about iterating upon what has
come before in a rapid and feedback-centric format.
Awesome. Not sure I dig the terminology given the etymological implications, but I'll roll with it.
To the extent that I can, I end each unit or project with reflection from my students both on their work and my implementation of that work as well. I try to word the questions in such a way that I get feedback for the same unit or project next year and so that I know what to try / not try in the following units or projects.
I'm about to stand behind two opposing ideas.
1) I've always wanted to hold school in a kind of drop-in way. The library we visited is the kind of space I've always seen this happening in. What if English was offered in sessions? Students needed a certain number of English units in a month, quarter, semester, etc. The faculty then designed curriculum. This could be one-off lectures, week- or month-long projects or term-long classes. Students then made their choice and participated. These would be the study rooms in the library. Other times, they would be in the common space, maybe meeting for group work, and dropping in on other disciplines' session in the same manner. In my mind, it would push the faculty to always bring their A-game and provide a source of data that could help with community planning in a way we don't do particularly well right now.
2) Students don't know what they don't know. They may here about a subject area and tune out to it because of association or because of a lack of association. Applying the approach above could mean students enter into a situation years down the road for which they are unprepared. I'm not going all E.D. Hirsch here. I'm just saying choice can lead to bad choices as easily as good ones.
That said, the adventurer in me wants to try #1.
I don't think that if #1 was employed it would mean that kids would be
unprepared. They might not know the “facts” or the “allusions” or the “type
of discourse” required for a given situation, but because they have created
their own learning path, they would know how to access that information. It
would also create a mentorship model that could be applied long-term. The
first model would allow kids to continue to come back to you even after
school ended and learn from you, or they could learn from one another. I
don't think it was expressed specifically, but the projects that they would
be working on would probably have a much better chance of “accidentally”
running into issues of discourse and diction than having to do a research
paper on a given subject when they will likely never have to do a research
paper outside of their academic life.
I want to start thinking of situations that would allow for the most kids to
grab on tight to something big enough and see it through. As I have said
before, I want kids to start businesses. I want kids to write books. I want
kids to design experiments. I want kids to write software. I want kids to
tell stories. I want kids to want these things and have a space to do it. I
don't want these projects to be “something extra”. I want them to be the
thing.
Sorry for so many quotations marks that aren't required. I'm not exactly
sure what the APA style is for a blog comment, do you?
I don’t think that if #1 was employed it would mean that kids would bernunprepared. They might not know the “facts” or the “allusions” or the “typernof discourse” required for a given situation, but because they have createdrntheir own learning path, they would know how to access that information. Itrnwould also create a mentorship model that could be applied long-term. Thernfirst model would allow kids to continue to come back to you even afterrnschool ended and learn from you, or they could learn from one another. Irndon’t think it was expressed specifically, but the projects that they wouldrnbe working on would probably have a much better chance of “accidentally”rnrunning into issues of discourse and diction than having to do a researchrnpaper on a given subject when they will likely never have to do a researchrnpaper outside of their academic life.rnrnI want to start thinking of situations that would allow for the most kids torngrab on tight to something big enough and see it through. As I have saidrnbefore, I want kids to start businesses. I want kids to write books. I wantrnkids to design experiments. I want kids to write software. I want kids torntell stories. I want kids to want these things and have a space to do it. Irndon’t want these projects to be “something extra”. I want them to be thernthing.rnrnSorry for so many quotations marks that aren’t required. I’m not exactlyrnsure what the APA style is for a blog comment, do you?
Pingback: Are We Actually Creating Startups Through Business Plan Competitions? « Robert Shedd
Pingback: Groundhog Day, or, the Problem with A/B Testing | Securing your world
Pingback: Groundhog Day, or, the Problem with A/B Testing | Securing your world
Pingback: Travel in the sky › Groundhog Day, or, the Problem with A/B Testing
Pingback: Groundhog Day, or, the Problem with A/B Testing | I love airchina
At our place, we began with offering one course which we knew was popular (courtesy of google) and at the end of the course, we asked students (via a feedback form) which courses they would be interested in next (with an option of specifying their own as well). If enough students opt for a particular course, we teach that next.
I completely agree with you. I really like this article. It contains a lot of useful information. I can set up my new idea from this post.