System Design And Analysis Complete

What is System Design

Systems design is the process of defining the architecture, product design, modules, interfaces, and data for a system to satisfy specified requirements. Systems design could be seen as the application of systems theory to product development

How to Prepare for System Design

While every system design interview is different, there are some common steps you should cover, even if the conversation might not be as sequential as your ideal thought process.

The system design interview is an open-ended conversation, which you’ll be expected to lead. Try using the following steps to guide your discussion:

Step 1 — Understand the Goals

Clarifying ambiguities early in the interview is critical. Candidates who spend time clearly defining the end goals of the system have a better chance of success than those who don’t. For that reason, make sure you understand the basic requirements and ask clarification questions. Start with the most basic assumptions:

·         What is the goal of the system?

·         Who are the users of the system? What do they need it for? How are they going to use it?

·         What are the inputs and outputs of the system?

Even if you’re asked about a well-known product, you should still share your assumptions about it with your interviewer. You may find that you and your interviewer don’t have the same assumptions about products like Twitter, Facebook, or Reddit. In addition to helping you focus, it also demonstrates product sensibility and good teamwork.

Step 2 — Establish the Scope

Now that you understand the system, try to describe the feature set that you’ll be talking about. Try to define all the features that you think of by their importance to the user. You don’t have to get it right on your first attempt, but make sure that you and your interviewer agree.

Ask clarifying questions, such as:

·         Do we want to discuss the end-to-end experience or just the API?

·         What clients do we want to support (mobile, web, etc)?

·         Do we require authentication? Analytics? Integrating with existing systems?

Take a few minutes to discuss this with your interviewer, and write it down.

Step 3 — Design for the Right Scale

The same feature set requires a very different approach for different scales. It’s important to determine the scale so that you know whether your data can fit on one machine or whether you need to scale the reads. You might ask:

·         What is the expected read-to-write ratio?

·         How many concurrent requests should we expect?

·         What’s the average expected response time?

·         What’s the limit of the data we allow users to provide?

Different answers require very different designs, so getting the scale right is key to success.

Step 4 — Start High-Level, then Drill-Down

Start with covering the end-to-end process, based on the goals you’ve established. This might include detailing different clients, APIs, backend services, offline processes, network architecture, data stores, and how they all come together to meet the requirements.

This is also a good point to identify the system’s entry-points, such as:

·         User interaction

·         External API calls

·         Offline processes

This will allow the conversation to drill down into potential performance bottlenecks, and decisions about the separation of responsibilities. Whichever approach you choose to start with, remember to always start simple, and iterate.

Step 5 — Data Structures and Algorithms (DS&A)

Wait, this again? Yes! Turns out, this is actually important in designing software systems.

URL shortener? Makes me think of a hashing function. Oh, you need it to scale? Sharding might help. Concurrency? Redundancy? Generating keys becomes even more complicated. Same goes for designing an analytics system, a news feed, or a Q&A forum, each with its own set of common DS&A.

Don’t forget to account for your scaling requirements, where analyzing runtime and memory complexity really becomes handy.

Step 6 — Tradeoffs

Almost every decision will involve a trade off. Being able to describe them in real time, as you’re suggesting solutions, shows that you understand that complex systems often require compromises and allow you to demonstrate your knowledge regarding the pros and cons of different approaches. Since there’s no one correct answer, having this discussion will give your interviewer the impression that you’re practical and will use the right tool for the job.

A few common examples might include:

·         What type of database would you use and why?

·         What caching solutions are out there? Which would you choose and why?

·         What frameworks can we use as infrastructure in your ecosystem of choice?

 

 

Practice is Key

While you can definitely hone down the theory by yourself, the last piece of the puzzle is practice. Make sure to apply these steps time and time again, answering questions about modern system design with real peers.

It’s the combination of knowledge, technique, and practice, that’ll enable you to land your dream job.

Additional Resources

Articles/Blogs:

·         Exponent’s System Design Interview Prep Course — An online course with mock interview videos and lessons to help you ace the system design interview

·         System Design Cheat Sheet — A guide on the key topics within system design

·         The Log: What every software engineer should know about real-time data’s unifying abstraction — A lengthy article about logs and tradeoffs

·         High Scalability — A blog about how real-world systems are designed

·         The Architecture of Open Source Applications (Volume 2): Scalable Web Architecture and Distributed Systems — An article about some of the key issues to consider when designing large websites

·         Distributed Systems Reading List — A list of articles on distributed systems to help change the way you think

Videos:

·         System Design Mock Interview: Instagram — YouTube walkthrough of a System Design mock interview about designing Instagram

·         System Design Mock Interview: Facebook Messenger — YouTube walkthrough of a System Design mock interview about designing Facebook

·         Intro to Architecture and Systems Design Interviews — YouTube tutorial made by an ex-Facebook engineer about system design interviews

·         System Design Introduction For Interview. — YouTube tutorial made by Tushar Roy about system design interviews

Top System Design Question

 

 How would you design a tinyURL system?

A tinyURL is an URL service that allows users to enter a long URL, and then it returns a shorter, unique URL. A hiring manager might ask this to allow you the opportunity to show your solid foundation in design. You can focus on other basics not listed in the example response, like how you create a unique ID for each URL, how you handle redirects and how you delete expired URLs.

Example: "When I was working for a public instant messaging site, I was charged with creating a simple system where every message was limited to 140 characters. It also necessitated shortened URLs of about 30 characters. This tinyURL system is also useful when entering hyperlinks in e-mails or on a smartphone, where there is room for error. TinyURL is a perfect example of the hashtag table. This data structure associates keys with values and is a simple connections code. By using this basic 16-bit hash table, I was able to optimize usability and meet the needs of the system."

How would you design a search engine?

Sometimes search engines are needed within a specific department of a company to systematically locate an item or important employee information. Hiring managers want to see that you can tailor designs to the needs of the company. You can detail some of the overall architecture and explain it, using the foundation below. You can also consider discussing any other relevant issues such as website front-end performance, testing search engine improvements and integrating previous search data and trends in indexing.

Example: "Before I relocated here, I was working on a project similar to this one. The search engine I had been enlisted to create needed to work with keyword searches. I began by building an indexer, which is a piece of software that crawls and produces results in a data structure. The crawler would put web page links together and group them or dump them into sets. Then the indexer ran as part of a reduce job to single things out. For each website, the number of links was calculated and analyzed for presentation. I had the crawl set for H1 and H2, rather than H3s. Then I checked outbound links to avoid spammers. Lastly, I checked the serving results to verify that the design was working at optimal capacity and relevancy."

 How do you design a web crawler, and when should it be used?

A crawler is a program designed to visit other sites and read them for information. This information is then used to create entries for a search engine index. It is typically called a 'bot" or "spider." Be certain to show within your explanation that you know the intricacies of web crawling.

Example: "Although crawling the web is a challenging task, I have managed to build one for a previous project. The crawler scrapes data from a specific sector, in this case, the fashion industry. I needed to integrate a URL dispatcher, which is a server whose responsibility is to distribute seed URL to a multitude of servers. Next, the crawl supervisor passed the URL to bots using the designed messaging queue. The spider, the basis for any crawler, extracted the data from the web page and loaded it into my file system. Next, the extract, transform and load (ETL) cleaned up the content and reformatted it to store it into the database. In such a way, I was able to crawl the web looking for and organizing the information needed."

How do you design a shared drive?

Hiring managers ask this to explore algorithm basics and backgrounds. Before you begin, make sure you understand the purpose of the task. Knowing if the changes will be registered in real time, if locking will be necessary and if it needs to be naturally convergent will help you give a complete answer.

Example: "This system works on differential synchronization. It is keeping two or more copies of the same document synchronized with each other in real time, so if a change is made on one version, the same alteration happens on all the others. It is a complex challenge, but differential synchronization is scalable and fault tolerant. The three common approaches are ownership, event passing and three way merges. I last had to do this to support in-house document sharing for one of our clients. They wanted real-time collaboration, so three-way merging was not a good option since changes are lost and cannot take effect, as major collisions are common. I used event-passing to allow for real-time collaboration as the locking or ownership approach would only allow the first one opening the document to make any adjustment. This served our client well, as its employees were able to work collaboratively even when out of office or on different schedules."

What is required to design a garbage collection system?

Garbage collection ensures a Java system is running appropriately and frees a programmer from having to do it manually. Hiring managers look to see if you know how to truly design the ins and outs of various systems. A GC makes systems memory efficient.

Example: "One of my recent clients needed a way to have more memory, but there was an issue with always having to go in and deal with memory deallocation. The nature behind garbage collection is to make a system appear as if it has a seemingly endless amount of memory. What is really happening is that the system is re-purposing the memory. When a system is running slowly, a garbage collector goes in and collects what is no longer being used. I set up their system so that if an object is referenced or recursive in nature, it remains. Next, it goes through methodically and marks whatever has not been referenced and sweeps only that. Using the mark and sweep method with the void command helps to repurpose and open up memory no longer being used. With this in place, my client had a faster system with less maintenance required."

 How do you design a recommendation system?

Recommendation systems help users find what they want more efficiently. They help clients and customers by offering alternatives and allowing for choice. Hiring managers inquire about this to see if you are able to create systems that are user-friendly and focused.

Example: "One of my first and most loyal clients had a problem where their customers were struggling to find options on their website. Their search had to be exact in order to find the product. I suggested we implement a recommendation system to help with customer satisfaction and possibly sales. Using the most prominent approach of collaborative filtering, I designed the system to weave a sort of information tapestry to give our client's customers suggestions based on user similarity. The system became more user-friendly and produced a 10% increase in sales for my client."

Q1) Design TinyURL or bit.ly (a URL shortening service)?

Answer:
Come up with features that the system should support first. Secondly, come up with the estimated numbers of how scalable the system can be.

While designing a service, there are three things to consider:

• API(REST API)-The client’ss communication approach with the service and a load balancer is the service’s front end.
• Application Layer – worker threads or hosts that take the URL and generate the tiny URL for it and store both of them in the persistence layer.
• Persistence Layer – Database

Things to analyze:

• Generate a unique ID for each URL and generate ID’s at scale since 1000’s of URL shortening requests come every second.
• Service handle redirects.
• Support custom URLs.
• Track click stats
• Delete expired URLs

2) Design YouTube/Netflix (a global video streaming service)?

Answer:
Things to analyze:

• In videos, the service will be storing and transmitting a huge amount of data that a large number of users can watch and share them simultaneously.
• Record statistics about videos, for example, the overall number of views, up-votes/down-votes, etc.
• Adding comments on videos in real-time.

Components:

• OC – Clouds like AWS, OpenConnect which act as a content delivery network.
• Backend – Database
• Client – Any device(Desktop, Android, iPhone) from which you play the video on YouTube/Netflix.

Let us move to the next System Design Interview Questions.

3) Designing Quora/Reddit/HackerNews (a social network + message board service)?

Answer:
Things to analyze:

The people who use the services can share links or post questions. Other users can answer questions or comment on the shared links. So the service should do the following:

• Records stats for each answer, e.g. the overall number of views, up-votes/down-votes, etc.
• Users should be in a position to follow other users or topics
• List of top questions on the timeline from all the users and the topics they follow (similar to newsfeed generation).

4) Design Facebook Messenger or WhatsApp (a global chat service)?

Answer:
This is the basic System Design interview questions asked in an interview. Things to analyze:

• Design one on one conversations between users.
• Extending your design to support group chats.
• What can be done when the user is not connected to the internet?
• When to send push notifications?
• How to provide end to end encryption?

Structure:

Client > Load balancer > Nodes > Cache(Redis) > Database(Casandra)

5) Design Search Typeahead(Autocomplete)?

Answer:
Things to analyze:

• Typeahead suggestions to be provided.
• Criteria for choosing the suggestions.
• Does the system need to be realtime?
• Support personalization with the suggestions.
• Queries per second to be handled by the system.
• Amount of data to be stored.

Part 2 – System Design Interview Questions (Advanced)

Let us now have a look at the advanced System Design Interview Questions.

6) Design Dropbox/Google Drive/Google Photos?

Answer:
Things to analyze:

• How to upload/view/search/share files or photos?
• Track permissions for file sharing.
• Allowing multiple users to edit the same document.

7) Design Facebook, Twitter or Instagram?

Answer:
Features to be considered:

• Some of the Twitter/Facebook/Instagram features to be supported.
• Need to support replies to tweets/grouping tweets by conversations.
• Privacy controls around each tweet.
• Supporting trending tweets
• Direct messaging
• Mentions/Tagging

Things to analyze:

• The number of users and traffic to be handled by the system.
• Amount of followers the user has.
• The number of times the tweet has been favourited.

Components required to be designed:

• Newsfeed generation
• Social graph
• Systematic search and storage for posts/tweets.

Let us move to the next System Design Interview Questions.

8) Design a Web Crawler?

Answer:
Scalable service is required that can crawl the entire web and can collect hundreds of millions of web documents.

Things to analyze:

• Finding new web pages.
• Prioritizing web pages that change dynamically.
• Ensuring that the crawler is not boundlessly attached to the same domain.

9) Design Uber:

Answer:
This is the frequently asked System Design interview questions in an interview. Architecture: Monolithic/Micro services(Real-time service, Front-end(Application) and database)

• The backend is servicing the mobile phone traffic, and clients connect to the backend over mobile data.
• Clients link to the dispatch system, which matches drivers and riders.
• Dispatch is written almost entirely in node.js.
• Maps/ETA: In order for dispatch to make an intelligent choice, it is important to get maps and routing data.
• Services: Business logic services mostly written in python.
• Databases: Postgres, Redis, MySQL.

Things to analyze:

• Customer requesting a ride and how to economically match them with the nearby drivers.
• Storing tons of geographical locations for drivers and riders who are always on the move.
• Handling updates to driver locations.

10) Design an API Rate Limiter(Github)?

Answer:
Things to analyze:

• Limiting the number of requests an entity can send to an API within a time window, for example, ten requests per second.
• Rate limiting should work for a scattered setup, as the APIs are reachable through a group of servers.

1) What is System Design?

System design is a process of defining the elements of a system such as the architecture, components, modules, and various interfaces.

2) What are the three most essential skills of system designer?

Important skills for system designer are:

• User interaction
• External API call
• Offline processes

3) How to design traffic control software?

To design a system for the traffic control system, as a software engineer, you need to make sure you know how to transition from one state to another. For example, Red to Green and from Green to Orange to Red, etc.

4) What is the benefit of a designing system like Pastebin?

Pastebin helps you design a system to paste code or text. You can share a link to that code anywhere you want. It’s not an online code editor, yet you can use this, a tool to store any text.

5) As a system designer, how you can design a universal file sharing and storage apps like Google Drive or Dropbox?

The above mention apps are used to store and share files, photos, and other media. We can design things like allowing users to upload/search/view files or photos. It checks permissions for file sharing and enables multiple users to make changes in the same document.

6) How can you design an ATM system?

An ATM helps a user to deposit and withdraw money. It also allows users to see their account balance. You need to make a design plan to create this system.

7) How can you design a web crawler like Google?

A web crawler visits your website and crawls all your page links and indexes so that it appears in a Google search result.

A crawler should be used for searching a specific file in a set of directories. Designing such a system requires lots of research and time.

8) What are the things you kept in mind while Designing a web crawler tool?

To design this, you need to analyze:

• Finding new web pages.
• Prioritizing web page which changes dynamically
• You also need to ensure that crawler shouldn’t be boundlessly attached to the same domain.

9) How can approach to design global cab services like Uber or Ola?

Uber and Ola are two widely used cab provider. It establishes communication between drivers and passengers. You need to create a module for GPS, rate calculation, current location, driver, and client information.

10) How do you approach to design a message board service sites?

HackerNews, Reddit, Quora are some of the most popular social network sites where users can post questions or share links. This type of system allows other users to answer questions or comment on the shared links.

11) What is the benefit of designing an application like Airbnb?

It helps you to upload rooms for rent and other users to rent them. Other vital features are for

• Publishers
• Admins
• Subscribers

12) How can you design an API Rate Limiter?

Here are the points that you need to keep in mind while designing an API Rate Limiter system:

• Limit the requests which any user can send to an API within a time window, i.e., 15 requests per second.
• The rate-limiting should work according to distributed setup because the APIs are accessible only for a cluster of servers.
• You need to make sure how you would handle throttling.

13) How do you approach system design?

Here are things that you need to keep in mind while designing this system:

• Helps users to search nearby friends or places.
• Check the ranking of place based on distance and user reviews.
• Store location data according to the population of the density of that area.

14) What are the important structured tools?

Important structure tools are:

1) Data Flow Diagrams, 2) Data Dictionary, 3) Decision Trees, 4) Structure English, and 5) Pseudocode.

15) In System Design process, what is Requirements Determination?

A requirement is most important for a new system which includes processing or capturing of data, controlling the activities of a business, producing information and supporting the management.

Requirement determination helps you to study the existing system and to gather details to find out what are the requirements, how it works, and what kind of improvements should be made.

16) How can you design a Twitter Clone?

Twitter is the most popular messaging service which allows you to broadcast message to all the people who are following you.

When you tweet, your follower able to read those messages, they can also retweet or like. To design such type of apps, you should include standard features like followers, tweet, hashtag, etc.

17) How can you design autocomplete functionality?

Here are important things for developing autocomplete functionality:

• Typeahead suggestion to be provided.
• Queries per second handled by the system.
• Support personalization with the suggestions.
• Amount of data to be stored.

18) Which is the primary tool used for structured Design?

Structure charts is a primary tool used for structured Design.

19) What are the important aspects of the System Study?

System study is essential to design any system.

Three most important aspect of System Study are:

• Identifying current issues and establishing new goals.
• Study of an existing system.
• Documenting the existing system.

20) What is the Step by step process to solve specific issues called?

This process is known as an algorithm which plays a significant part for system designing.

21) What is the approach used in top-down analysis and Design?

To approach top-down analysis, you need to identify a top-level function then create a hierarchy of lower-level module and components.

22) Explain the term controller

A controller is a program component which helps you to make decisions and directs other components.

23) Which of the following is not a factor in the failure of the system?

The size of the organization can’t be considered as a factor for system development and designing projects.

24) Documentation should be prepared on which state?

Documentation should be prepared at every stage of system designing.

25) In the system design process, where is problem analysis done?

Problem analysis is done at the systems analysis phase.

26) What do you know about the app booking app like book my show?

Book my show allows users to book their ticket for shows, events, movie, or sports.  It will enable them to pay and get a refund of movie tickets.

27) What are the Types of Documentation in System Design?

Four types of documentation are:

• Program documentation
• System documentation
• Operations documentation
• User documentation

 

 

 

Tips for any SDI Question

• Start each problem by stating what you know: List all required features of the system, common problems you expect to encounter with this sort of system, and the traffic you expect the system to handle. The listing process lets the interviewer see your planning skills and correct any possible misunderstandings before you begin the solution.

• Narrate any trade-offs: Every system design choice matters. At each decision point, list at least one positive and negative effect of that choice.

• Ask your interviewer to clarify: Most system design questions are purposefully vague. Ask clarifying questions to show the interviewer how you’re viewing the question and your knowledge of the system’s needs.

• Discuss emerging technologies: Conclude each question with an overview of how and where the system could benefit from machine learning. This will demonstrate that you’re not just prepared for current solutions but future solutions as well.