JavaScript is disabled. Please enable Javascript for full website functionality. To learn how to enable Javascript visit http://www.enable-javascript.com/

The Connector - Spring 2021 Issue

Stella® Architect Powers Management Experimentation at MIT

Hazhir Rahmandad Hazhir Rahmandad

“In the last decade, online behavioral experiments have become more common,” says Hazhir Rahmandad, Associate Professor of System Dynamics, Sloan School of Management, Massachusetts Institute of Technology. “We used to have to recruit people and bring them into a lab but, using services like Amazon Mechanical Turk and Stella, we can quickly find subjects who participate in the experiment at home. It’s less costly, more efficient and I’ll be leveraging the technology more and more.”

Rahmandad found his way to Stella three years ago after trying to create an online experiment using another platform. “My graduate student worked on the project for six months but it still wasn’t finished when he graduated,” says Rahmandad. “I came across Stella Architect and finished the project in two days.” Now, Rahmandad and his colleagues have completed two experiments that study managerial decision-making and created a model that describes COVID-19 pandemic response across 92 countries.

Working with Michael Shayne Gary, PhD, Associate Professor, Australian School of Business, University of New South Wales, Rahmandad used Stella Architect to study how managers apply, or don’t apply, learning to strategic decision-making. “We created a model of a service operation and presented participants with two choices to manage this simulated operation: how much to pay their employees and how to structure the work,” says Rahmandad. “The system includes two promising, but incompatible, strategies.”

The first strategy calls for hiring people into low paying, narrowly defined positions. Training takes less time, there is little room for confusion or mistakes, but employee dissatisfaction and turnover is high. The second strategy hires high quality employees into higher paid positions that engage them in more complex tasks. Training takes longer but employee satisfaction is high and turnover is relatively low. These mimic low-roads and high-roads strategies in many service operations.

“We wondered why we don’t see more instances of businesses using the high-roads strategy,” says Rahmandad. “Our hypothesis was that delays between adopting a strategy and realizing the full benefit or downside of that action influenced what gets adopted.”

Amazon Mechanical Turk, an online outsourcing marketplace that connects employers to individuals who can complete repetitive tasks or, in this case, subjects to experimenters, was used to recruit non-expert study participants. Subjects with managerial experience were recruited through Qualtrics, an online experience management platform.

Using a Stella Architect simulation of the business, published to the isee Exchange, both groups of subjects attempted to find the combination of work structure and compensation that maximized profits through four training rounds of 24 quarters each. They received feedback through a simulator dashboard that tracks quarterly metrics for key performance indicators.

tracking performance Subjects were able to track the performance of their strategy decisions.

Published in Organization Science in December 2020 (“Delays Impair Learning and Can Drive Convergence to Inefficient Strategies”), the study showed that delays and uncertainty influence strategic decision-making for both non-experts and experienced managers. “We noted that the longer the delay in seeing benefit from the chosen strategy, the more likely subjects were to move towards a different strategy with faster payback,” says Rahmandad. “In this study, we used the simulation environment to look at managerial learning in the presence of feedback delays. Business simulations could also compress the time needed to learn about the impact of alternative strategies, which otherwise may take years to unfold, helping leaders identify and internalize strategies that pay off over the longer term.”

The second experiment, conducted with colleagues Jerker Denrell, PhD, University of Warwick and Drazen Prelec, Phd, Massachusetts Institute of Technology, also leveraged a Stella model on the isee Exchange and Amazon Mechanical Turk, this time to investigate learning in simpler dynamic tasks, including in settings with multi-player markets. Similar to the first study, Rahmandad and his colleagues sought to determine why most decision makers have trouble making near-term decisions that benefit long-term performance.

“We were able to build a simple model and simulation based on a task called Harvard Game, recruit subjects, and start collecting data in days,” says Rahmandad. Using the online, multiplayer simulation, subjects repeatedly chose between two options (buttons). A payback, or profit, was determined by the combination of choices they made and the choices that other players were making.

“We know it’s harder to learn to make best choices when the task is dynamic,” says Rahmandad. “We found that the challenge persisted even when everything about the task structure was fully known and participants were given plenty of hints. In fact, many continued to learn the exact wrong lesson.”

profitability feedback This interface provided subjects with feedback measuring the profitability of their prior 10 choices.

By varying the type of information provided to subjects, the study showed that unprofitable strategies could not be attributed to game complexity, ambiguities, or learning difficulty. Rather, Rahmandad and his colleagues found that the biggest stumbling block was the ability to think systematically and understand how trade-offs influence outcomes over time. Their experiment and findings were published in Strategic Management, October 2020 (“What Makes Dynamic Strategic Problems Difficult? Evidence from an experimental study”).

“I’ll continue to leverage the work we’ve done with Stella Architect and online subject pools to understand dynamic learning and decision-making,” says Rahmandad. “Through the experiments we’ve conducted so far, we have strived to build one-click solutions that move us from recruiting subjects to running the experiment to processing data in hours. That provides huge advantages in trying alternative ideas and coming up with better research designs in a matter of days.”

The ability to quickly generate and run experiments and simulations enables Rahmandad to investigate and disseminate current issues quickly. In July 2020, he and MIT colleagues TY Lim, a PhD student and John Sterman, PhD, Forrester Professor of Management Science Director, System Dynamics Group Chair, published “Behavioral Dynamics of COVID-19: Estimating Under-Reporting, Multiple Waves, and Adherence Fatigue Across 92 Nations” on the Social Science Research Network.

“We calibrated our model using VenSim and then moved it to Stella Architect with a user interface,” says Rahmandad. “We wanted to share broadly how changes in behavior in response to risk of COVID-19 and different vaccination rollouts create the disease trajectories in the coming months.”

COVID-19 behavior model Overview of COVID-19 behavior model

The model provides a global view of a global crisis, including essential biological and behavioral mechanisms and feedback loops. It explains the significant differences in per-capita cases and death rates across countries and offers projections for both measures. Not surprisingly, collected data indicates that diminishing adherence to COVID-19 precautions added significantly to cumulative cases and that improvements in adherence would reduce both infection and death rates with limited additional economic costs.

The simulator is posted on the isee Exchange at: https://exchange.iseesystems.com/public/mitsdl/covidglobal/.

“isee systems’ support has been really helpful through all these experiments and simulations, even adding requested features in a matter of weeks,” says Rahmandad. “Whenever I need to run an online experiment or simulation, I expect to start with Stella Architect.”

James Madison University Integrates Sciences with Stella Architect and Stella Professional

Rod MacDonald Rod MacDonald

Integrated Science and Technology (ISAT) majors at James Madison University tackle some pretty tough problems. How can dairy farming be made sustainable? Does vaping actually pose less health risk than smoking tobacco? How can North Carolina hog farmers reduce pollution?

To understand and find solutions to those and other real, complex problems, students integrate coursework in biology, chemistry, environmental science, engineering, mathematics, and physics with the social contexts to test hypotheses using real-world data. “We use system dynamics to integrate the subject areas across the major,” says Rod MacDonald, PhD, Assistant Professor, ISAT. “Stella Architect and Stella Professional have become our tools of choice.”

While other undergraduate programs apply system dynamics within specific courses, the James Madison University program is unique in using system dynamics to integrate a range of science and technology subject areas. “A constant concept across all their courses of study is conservation of matter,” says MacDonald. “Anything that goes in to a biological system, chemical equation, or physics experiment has to be accounted for. That’s an essential concept of modeling. All stock units have to be correct and accounted for in their flows.”

“The 80+ credit-hour major is general but students specialize in environmental science, energy, manufacturing, biotechnology, and knowledge information management,” says MacDonald. “Students take a system dynamics course in their sophomore year and then engage in a two-course, year-long holistic problem-solving project in their junior year.”

This first course introduces students to modeling basics, including stocks, flows, and causal loops. “Students engage in actual model building but we know the model they’ll build,” says MacDonald. “They’re given a question with a known answer and we help them work towards that answer.”

Each year, the juniors split into small groups to investigate a topic in a predetermined problem area using Stella Architect and Stella Professional. “This year’s topic is Wholistic Problem Solving for the Food System,” says MacDonald. “The two-course year-long project is team taught by professors with different backgrounds and areas of expertise. In the first semester, students learn about their topic, think through the main causal loops and behavior of these loops over time, and develop a problem statement.

In the second semester, students build models that describe the system that includes their topic, take a deep dive into available data, create simulations to test hypotheses, and discuss the topic’s social context, including the stakeholders, legal issues, and cultural issues. At the end of the semester, students write a paper and present their work.

Even though students have thoroughly researched their topic, models and simulations lead to new insights. “A group of students was looking at the transport of fresh fruits and vegetables,” says MacDonald. “They saw that shipping by plane was ridiculous in terms of fuel cost and carbon output and hypothesized that people should only eat local produce that is in season. They were surprised to see that, even when the system is improved for sustainability, plane transport remains important. Markets need fresh food to arrive fresh and if you shift to only local, in-season produce, agricultural economies in other places are adversely impacted.”

Loops the Matter™, a new feature in Stella Architect, keeps students focused on the most meaningful part of their complex models. “We look at Jay Forrester’s famous global economy World Dynamics model,” says MacDonald. “Even though it’s a complicated model, Loops that Matter highlights the most important points and students get really engaged. It’s a great way of hooking them into the system dynamics way of seeing the world.”

Integrated Sciences students Integrated Sciences students use Loops that Matter to identify and discuss essential portions of Forrester’s classic World Dynamics model.

Students rely on Loops that Matter as they build and refine their own complex models. “The software’s ability to indicate feedback loop strength by the thickness of the arrows and label them as reinforcing or balancing allows me to spend more time teaching about the meaning of loops rather than the mechanics of loops,” says MacDonald. “And it focuses them on the most important parts of the system they’re investigating.”

“Of course, I can tell them which part of their model is the most important, but they trust the software more than they trust me!” says MacDonald. “They click on the highlight loop button, see the most important feedback loops, and say, ‘Ok, we have to nail the data here.”

Students sharing their modeling work with one another and their professors is an important part of the process. “Creating a user interface for their model is easy with Stella Architect,” says MacDonald. “Students post their models on the isee Exchange™, and using the intuitive interfaces, walk through their models without having to explain the spaghetti underneath.”

Now in its 20th year, the ISAT major already attracts 100 James Madison University students each year. “Rather than developing niche expertise, ISAT majors graduate as Jacks of all Trades,” says MacDonald. “With a solid foundation in science and technology and the ability to apply system dynamics, they can go on to further study in any area or help solve any problem.”

New Course: Tracking Projects with Earned Value

isee systems has a new online course! Expand your project management knowledge beyond the Modeling Projects course to include traditional project planning techniques and earned-value metrics. Used in concert with the project model developed in that course, these allow a manager to handle the internal and external pressures that commonly affect projects. Beginning with the basics of modeling and earned-value analysis, learn how to produce a robust model of a project that corrects project behavior under severe stress and avoids excessive overtime. Join instructor Jack Nevison as he shares tips and tricks that he’s learned in more than 50 years of experience in the field.

Buy the course here.

Improved Software Updates

We recently changed the way we name our software updates to give you more information about available builds. Sometimes we identify bugs that can easily be fixed with a patch. Rather than wait for the next version update, we include the fix in a new build and republish it under the same version number. This means it’s possible to have the latest version and still be out of date. You will notice in your account that each version now has a build number in parentheses. If the software notifies you that a new version is available, but you already have the latest version number, check that the build number of your installed version matches the latest build number in your account. If they don’t, there’s been a patch since your last download, and you should download that version again to update.

To check your version and build number on Windows, select the Help menu, then “About Stella.” On Macintosh, select the Stella menu, then “About Stella.”

Story of the Month: Vaccines

The vaccination program in the United States and its effort to control the COVID-19 pandemic has led the news for some months. Americans have debated the effects of vaccine hesitancy, people's behavior in response to perceived risk, and vaccine effectiveness. In this story of the month, we explore these issues and see what impacts they have on the COVID-19 cases and deaths both historically and in the near future.

View it here, and contact us with any questions or comments.

New Interface Tutorials

We’ve added a new series of interface building tutorials that show you how to add a simple interface to a model. You’ll learn how to create new pages, templates, text boxes, graphics frames, action buttons, navigation bars, graphs, sliders and knobs. You can find them here.

On the Road...Sort Of

The 39th annual International System Dynamics Conference is coming soon and once again, we will attend virtually. We will maintain a booth where attendees can stop by to learn more about our software or can just come chat with us about their SD projects! Co-presidents Bob Eberlein and Karim Chichakly and Lead Developer Billy Schoenberg will be presenting parallel sessions on “Perfect Mixing in COVID Models," "Systems Thinking for Public Policy: A Case Study in Brazil," and "Improving Loops That Matter," respectively. Billy will also present a poster. We will also give four workshops ranging from building models and interfaces to teaching with Loops That Matter to modeling COVID-19. It is always a delight to connect and network with our customers and meet new users. We hope to see many of you at the conference!

The fourth annual South African Chapter System Dynamics Competition kicked off in early May with participants from all over the world. This year's challenge looks at a world after COVID-19. Using Stella Architect, participants must choose an area of interest and explore how the pandemic has changed it. The winners will be announced at the ninth annual South African System Dynamics Conference in November. We wish all the participants good luck!

Recent Posts

Fall 2020 Issue

A simulation modeler studies the spread of a water-born infection, undergraduate students use Stella® for socially-distanced research, remote learning policies, the Stella Users Network, version 2.0.2, and isee systems goes back on the virtual road...

Spring 2020 Issue

A social scientist uses system dynamics to improve early childhood education, a professor uses Stella to model the relationship between economy and environment, lead developer Billy Schoenberg shares the history of Stella's new Loops That Matter™ feature, the new Beer Game, isee systems' response to COVID-19, and isee systems goes (virtually) on the road...

Fall 2019 Issue

A PhD student designs a plan to save water with Stella®, a credit union CEO collaborates with a system dynamics expert to better help his customers, Jon Darkow shares Stella tips and tricks, the 2021 isee user conference, new online courses, the next Barry Richmond Award, and isee systems heads to Australia...

Phone: (603) 448-4990 Email: info@iseesystems.com

   Monday - Friday: 9:00 am - 5:00 pm EDT | Saturday - Sunday: Closed
Wheelock Office Park | 31 Old Etna Rd, Suite 7N | Lebanon, NH 03766 | US

isee systems inc. holds trademarks registered in the U.S. Patent and Trademark Office for iThink®, Stella®, isee systems® and claims the following trademarks; isee NetSim™, isee Exchange™, Stella Live™, Causal Lens™, Stella Online™ and Stella Professional Online™.

Terms of Use

© 2021. isee systems inc . All rights reserved.