Large software project challenges — dilemma and solution

Weekly Dev Blog
4 min readMay 19, 2021

Scaling is one of the main challenges in software development. The concept is totally different between serving 10,000 vs 10 concurrent users. Even if the functional requirement is the same for both case, the implementation will be totally different. Usually, when software scales beyond its original planned capacity, a refactor is required.

Large software project dilemma

A classic large software project usually follows the development timeline:

  • in the beginning, a small project (1–3 people), a simple project with limited function. After the first release, it obtains positive feedback from the user. The bugfix is quick due to the feasibility for user and developer to directly interact.
  • the business decides to invest in more staff for the project. The project grows but the software starts to become complex, development speed starts to slow down. Takes longer to bugfix and introduce new functionality.
  • the business decides to further invest in having more staff. However, the increased staff slows down the productivity of existing member. A common dilemma: as the team grows bigger, the average productivity decreases.
  • many years later, with the aging code and increasing complexity, the project becomes hard to move forward. In some extreme situation, the maintenance cost becomes too high and impossible to make changes
  • in the end, the whole project becomes a technical dept, waiting for a new project to replace it.

Why large project has poor development efficiency

A large software project is like a big elephant. It looked terrified but when moving, moves very slow. There are two reasons behind this: code complexity and team

Code complexity

As the number lines grow, it becomes difficult for an individual developer to understand the whole codebase. If the team has more than 5 people and each responsible for 1 function. Then it becomes almost impossible for an individual to track the progress of the whole system.

If a person joins the team, due to unable to comprehend the whole system working detail, the person…