- Parallel Programming with Python
- Jan Palach
- 381字
- 2021-07-16 11:22:19
Exploring common forms of parallelization
There is a certain confusion when we try to define the main forms of paralleling systems. It is common to find quotations on parallel and concurrent systems as if both meant the same thing. Nevertheless, there are slight differences between them.
Within concurrent programming, we have a scenario in which a program dispatches several workers and these workers dispute to use the CPU to run a task. The stage at which the dispute takes place is controlled by the CPU scheduler, whose function is to define which worker is apt for using the resource at a specific moment. In most cases, the CPU scheduler runs the task of raking processes so fast that we might get the impression of pseudo-parallelism. Therefore, concurrent programming is an abstraction from parallel programming.
The following diagram shows a concurrent program scheme:

Concurrent programming scheme.
Parallel programming can be defined as an approach in which program data creates workers to run specific tasks simultaneously in a multicore environment without the need for concurrency amongst them to access a CPU.
The following figure shows the concept of parallel systems:

Parallel programming scheme.
Distributed programming aims at the possibility of sharing the processing by exchanging data through messages between machines (nodes) of computing, which are physically separated.
Distributed programming is becoming more and more popular for many reasons; they are explored as follows:
- Fault-tolerance: As the system is decentralized, we can distribute the processing to different machines in a network, and thus perform individual maintenance of specific machines without affecting the functioning of the system as a whole.
- Horizontal scalability: We can increase the capacity of processing in distributed systems in general. We can link new equipment with no need to abort applications being executed. We can say that it is cheaper and simpler compared to vertical scalability.
- Cloud computing: With the reduction in hardware costs, we need the growth of this type of business where we can obtaining huge machine parks acting in a cooperative way and running programs in a transparent way for their users.
The following figure shows a distributed system scheme:

Distributed programming scheme.
- Python從小白到大牛
- Mastering Objectoriented Python
- Oracle Database In-Memory(架構與實踐)
- Python Deep Learning
- JavaScript前端開發與實例教程(微課視頻版)
- Android NDK Beginner’s Guide
- Android系統原理及開發要點詳解
- 現代C++編程實戰:132個核心技巧示例(原書第2版)
- Learning Hadoop 2
- 運維前線:一線運維專家的運維方法、技巧與實踐
- Java Web從入門到精通(第2版)
- Getting Started with hapi.js
- 編程真好玩:從零開始學網頁設計及3D編程
- 區塊鏈原理與技術應用
- Getting Started with SpriteKit