舉報 
會員
Mastering Concurrency in Python
Quan Nguyen 著
更新時間：2021-06-10 19:25:01
開會員，本書免費讀 >
Pythonisoneofthemostpopularprogramminglanguages,withnumerouslibrariesandframeworksthatfacilitatehigh-performancecomputing.ConcurrencyandparallelisminPythonareessentialwhenitcomestomultiprocessingandmultithreading;theybehavedifferently,buttheircommonaimistoreducetheexecutiontime.Thisbookservesasacomprehensiveintroductiontovariousadvancedconceptsinconcurrentengineeringandprogramming.MasteringConcurrencyinPythonstartsbyintroducingtheconceptsandprinciplesinconcurrency,rightfromAmdahl'sLawtomultithreadingprogramming,followedbyelucidatingmultiprocessingprogramming,webscraping,andasynchronousI/O,togetherwithcommonproblemsthatengineersandprogrammersfaceinconcurrentprogramming.Next,thebookcoversanumberofadvancedconceptsinPythonconcurrencyandhowtheyinteractwiththePythonecosystem,includingtheGlobalInterpreterLock(GIL).Finally,you'lllearnhowtosolvereal-worldconcurrencyproblemsthroughexamples.Bytheendofthebook,youwillhavegainedextensivetheoreticalknowledgeofconcurrencyandthewaysinwhichconcurrencyissupportedbythePythonlanguage
最新章節
- Leave a review - let other readers know what you think
- Other Books You May Enjoy
- Chapter 19
- Chapter 18
- Chapter 17
- Chapter 16
品牌：中圖公司
上架時間：2021-06-10 18:25:27
出版社：Packt Publishing
本書數字版權由中圖公司提供，并由其授權上海閱文信息技術有限公司制作發行
- Leave a review - let other readers know what you think 更新時間：2021-06-10 19:25:01
- Other Books You May Enjoy
- Chapter 19
- Chapter 18
- Chapter 17
- Chapter 16
- Chapter 15
- Chapter 14
- Chapter 13
- Chapter 12
- Chapter 11
- Chapter 10
- Chapter 9
- Chapter 8
- Chapter 7
- Chapter 6
- Chapter 5
- Chapter 4
- Chapter 3
- Chapter 2
- Chapter 1
- Assessments
- Further reading
- Questions
- Summary
- Debugging and concurrency
- Debugging tools and techniques
- Debugging concurrent programs
- Testing programs concurrently
- Static code analysis
- Unit testing
- Testing concurrent programs
- Testing and concurrency in Python
- Running on the cloud
- Executor pool
- Background scheduler
- Blocking scheduler
- Examples in Python
- Common scheduler methods
- Trigger keywords
- Executor classes
- Scheduler classes
- APScheduler API
- APScheduler functionalities
- Not a scheduling service
- Installing APScheduler
- Scheduling with APScheduler
- Technical requirements
- Testing Debugging and Scheduling Concurrent Applications
- Further reading
- Questions
- Summary
- Making the server non-blocking
- Asynchronous generators and the send method
- Generators in Python
- Analyzing the concurrency of the server
- Building a non-blocking server
- Implementing the calculator server
- The underlying calculation logic
- Building a calculator server with the socket module
- Building a simple echo server
- The API of the socket module
- The theory of server-side communication
- Low-level network programming via the socket module
- Technical requirements
- Building a Server from Scratch
- Further reading
- Questions
- Summary
- Simulation in Python
- Atomic versus nonatomic
- Innate atomicity in Python
- The GIL reconsidered
- What does it mean to be atomic?
- Atomic operations in Python
- In the context of concurrency
- Memory model as a labeled directed graph
- The components of Python memory manager
- Python memory model
- Technical requirements
- Memory Models and Operations on Atomic Types
- Further reading
- Questions
- Summary
- Building on simple data structures
- RCU as a solution
- Implementing a simple network data structure in Python and race conditions
- Introduction to the network data structure
- The impossibility of being lock-free in Python
- Mutex-free concurrent data structures in Python
- A few considerations for approximate counter designs
- Implementing approximate counters in Python
- The idea behind approximate counters
- Approximate counters as a solution for scalability
- Analysis of the scalability of the counter data structure
- The concept of scalability
- Embedding locks in the data structure of the counter
- LocklessCounter and race conditions
- Lock-based concurrent data structures in Python
- Technical requirements
- Designing Lock-Based and Mutex-Free Concurrent Data Structures
- Further reading
- Questions
- Summary
- Utilizing a different Python interpreter
- Getting around the GIL with native extensions
- Implementing multiprocessing rather than multithreading
- How to work with the GIL
- The potential removal of the GIL from Python
- Problems raised by the GIL
- The problem that the GIL addresses
- An analysis of memory management in Python
- An introduction to the Global Interpreter Lock
- Technical requirements
- The Global Interpreter Lock
- Further reading
- Questions
- Summary
- Networking
- Operating systems
- Security
- Race conditions in real life
- Locks do not lock anything
- Turning a concurrent program sequential
- The downside of locks
- Implementation in Python
- The effectiveness of locks
- Locks as a solution to race conditions
- Simulating race conditions in Python
- How race conditions occur
- Critical sections
- The concept of race conditions
- Technical requirements
- Race Conditions
- Further reading
- Questions
- Summary
- Solutions to starvation
- The third readers-writers problem
- The second readers-writers problem
- The first readers-writers problem
- Problem statement
- The readers-writers problem
- Starvation's relationship to deadlock
- Causes of starvation
- Scheduling
- What is starvation?
- The concept of starvation
- Technical requirements
- Starvation
- Further reading
- Questions
- Summary
- The concept of livelock
- Concluding note on deadlock solutions
- An additional note about locks
- Ignoring locks and sharing resources
- Implementing ranking among resources
- Approaches to deadlock situations
- Python simulation
- Deadlock in a concurrent system
- The Dining Philosophers problem
- The concept of deadlock
- Technical requirements
- Deadlocks
- Further reading
- Questions
- Summary
- Writing files asynchronously
- Fetching a website's HTML code
- Installing aiohttp and aiofiles
- Client-side communication with aiohttp
- Closing the transports
- Sending messages back to clients
- Simulating a connection channel
- Installing Telnet
- Starting a server
- Python example
- The big picture of asyncio's server client
- Transports and protocols in asyncio
- Asynchronous programming for communication channels
- Communication protocol layers
- The ecosystem of communication channels
- Technical requirements
- Building Communication Channels with asyncio
- Further reading
- Questions
- Summary
- Examples in Python
- Changes in the framework
- concurrent.futures as a solution for blocking tasks
- Inherently blocking tasks
- Improvements from Python 3.7
- Asynchronous prime-checking
- A note about blocking functions
- Asynchronously counting down
- The asyncio framework in action
- Asyncio API
- Coroutines event loops and futures
- The asyncio module
- Technical requirements
- Implementing Asynchronous Programming in Python
- Further reading
- Questions
- Summary
- An example in Python
- Asynchronous versus threading and multiprocessing
- Asynchronous versus synchronous programming
- Asynchronous versus other programming models
- A quick analogy
- Technical requirements
- Introduction to Asynchronous Programming
- Further reading
- Questions
- Summary
- Processing input/output concurrently
- Spawning an appropriate number of processes
- Choosing the correct way (out of many)
- Good concurrent image processing practices
- Applying concurrency to image processing
- Thresholding
- Grayscaling
- Image processing techniques
- OpenCV API
- Coordinates inside an image
- Pixels and image files
- RGB values
- Computer image basics
- Installing OpenCV and NumPy
- Python as an image processing tool
- Image processing fundamentals
- Technical requirements
- Concurrent Image Processing
- Further reading
- Questions
- Summary
- Real-life applications of concurrent reduction operators
- Example implementation in Python
- Examples and non-examples
- Properties of a reduction operator
- The concept of reduction operators
- Technical requirements
- Reduction Operators in Processes
- Further reading
- Questions
- Summary
- Message passing between several workers
- Message passing for a single worker
- Interprocess communication
- Terminating processes
- Waiting for processes
- Determining the current process
- Determining the current process waiting and terminating processes
- The Pool class
- The process class
- An overview of the multiprocessing module
- Introductory example in Python
- Multiprocessing
- Processes versus threads
- The concept of a process
- Technical requirements
- Working with Processes in Python
- Further reading
- Questions
- Summary
- Avoid making a large number of requests
- Update your program regularly
- Error handling
- Consider the terms of service and data-collecting policies
- Good practices in making web requests
- Timeout specifications
- Support from httpstat.us and simulation in Python
- The problem of timeout
- Refactoring request logic
- Spawning multiple threads
- Concurrent web requests
- Running a ping test
- Making a request in Python
- The requests module
- HTTP status code
- HTTP requests
- HTML
- The basics of web requests
- Technical requirements
- Concurrent Web Requests
- Further reading
- Questions
- Summary
- Example of deadlock handling
- The with statement in concurrent programming
- The syntax of the with statement
- The with statement as a context manager
- Starting from managing files
- Context management
- Technical requirements
- Using the with Statement in Threads
- Further reading
- Questions
- Summary
- Multithreaded priority queue
- Queuing in concurrent programming
- The queue module
- A connection between real-life and programmatic queues
- Multithreaded priority queue
- An example in Python
- The threading.Lock class
- The concept of thread synchronization
- Synchronizing threads
- Starting a thread with the threading module
- Starting a thread with the thread module
- Creating a new thread in Python
- The threading module in Python 3
- The thread module in Python 2
- An overview of the threading module
- An example in Python
- Multithreading
- Threads versus processes
- The concept of a thread
- Technical requirements
- Working with Threads in Python
- Further reading
- Questions
- Summary
- Practical applications of Amdahl's Law
- How to simulate in Python
- Amdahl's Law's relationship to the law of diminishing returns
- Implications
- A quick example
- The formula for Amdahl's Law
- Formula and interpretation
- Terminology
- Amdahl's Law
- Technical requirements
- Amdahl's Law
- Further reading
- Questions
- Summary
- Downloading example code
- General setup
- Setting up your Python environment
- Why Python?
- A brief overview of mastering concurrency in Python
- The future
- The present
- The history of concurrency
- The history present and future of concurrency
- I/O bound
- Example 2 – inherently sequential tasks
- Inherently sequential
- Embarrassingly parallel
- Not everything should be made concurrent
- A quick metaphor
- Concurrent versus parallel
- Example 1 – checking whether a non-negative number is prime
- Concurrent versus sequential
- What is concurrency?
- Technical requirements
- Advanced Introduction to Concurrent and Parallel Programming
- Reviews
- Get in touch
- Conventions used
- Code in Action
- Download the color images
- Download the example code files
- To get the most out of this book
- What this book covers
- Who this book is for
- Preface
- Packt is searching for authors like you
- About the reviewers
- About the author
- Contributors
- Packt.com
- Why subscribe?
- About Packt
- Dedication
- Title Page
- coverpage
- coverpage
- Title Page
- Dedication
- About Packt
- Why subscribe?
- Packt.com
- Contributors
- About the author
- About the reviewers
- Packt is searching for authors like you
- Preface
- Who this book is for
- What this book covers
- To get the most out of this book
- Download the example code files
- Download the color images
- Code in Action
- Conventions used
- Get in touch
- Reviews
- Advanced Introduction to Concurrent and Parallel Programming
- Technical requirements
- What is concurrency?
- Concurrent versus sequential
- Example 1 – checking whether a non-negative number is prime
- Concurrent versus parallel
- A quick metaphor
- Not everything should be made concurrent
- Embarrassingly parallel
- Inherently sequential
- Example 2 – inherently sequential tasks
- I/O bound
- The history present and future of concurrency
- The history of concurrency
- The present
- The future
- A brief overview of mastering concurrency in Python
- Why Python?
- Setting up your Python environment
- General setup
- Downloading example code
- Summary
- Questions
- Further reading
- Amdahl's Law
- Technical requirements
- Amdahl's Law
- Terminology
- Formula and interpretation
- The formula for Amdahl's Law
- A quick example
- Implications
- Amdahl's Law's relationship to the law of diminishing returns
- How to simulate in Python
- Practical applications of Amdahl's Law
- Summary
- Questions
- Further reading
- Working with Threads in Python
- Technical requirements
- The concept of a thread
- Threads versus processes
- Multithreading
- An example in Python
- An overview of the threading module
- The thread module in Python 2
- The threading module in Python 3
- Creating a new thread in Python
- Starting a thread with the thread module
- Starting a thread with the threading module
- Synchronizing threads
- The concept of thread synchronization
- The threading.Lock class
- An example in Python
- Multithreaded priority queue
- A connection between real-life and programmatic queues
- The queue module
- Queuing in concurrent programming
- Multithreaded priority queue
- Summary
- Questions
- Further reading
- Using the with Statement in Threads
- Technical requirements
- Context management
- Starting from managing files
- The with statement as a context manager
- The syntax of the with statement
- The with statement in concurrent programming
- Example of deadlock handling
- Summary
- Questions
- Further reading
- Concurrent Web Requests
- Technical requirements
- The basics of web requests
- HTML
- HTTP requests
- HTTP status code
- The requests module
- Making a request in Python
- Running a ping test
- Concurrent web requests
- Spawning multiple threads
- Refactoring request logic
- The problem of timeout
- Support from httpstat.us and simulation in Python
- Timeout specifications
- Good practices in making web requests
- Consider the terms of service and data-collecting policies
- Error handling
- Update your program regularly
- Avoid making a large number of requests
- Summary
- Questions
- Further reading
- Working with Processes in Python
- Technical requirements
- The concept of a process
- Processes versus threads
- Multiprocessing
- Introductory example in Python
- An overview of the multiprocessing module
- The process class
- The Pool class
- Determining the current process waiting and terminating processes
- Determining the current process
- Waiting for processes
- Terminating processes
- Interprocess communication
- Message passing for a single worker
- Message passing between several workers
- Summary
- Questions
- Further reading
- Reduction Operators in Processes
- Technical requirements
- The concept of reduction operators
- Properties of a reduction operator
- Examples and non-examples
- Example implementation in Python
- Real-life applications of concurrent reduction operators
- Summary
- Questions
- Further reading
- Concurrent Image Processing
- Technical requirements
- Image processing fundamentals
- Python as an image processing tool
- Installing OpenCV and NumPy
- Computer image basics
- RGB values
- Pixels and image files
- Coordinates inside an image
- OpenCV API
- Image processing techniques
- Grayscaling
- Thresholding
- Applying concurrency to image processing
- Good concurrent image processing practices
- Choosing the correct way (out of many)
- Spawning an appropriate number of processes
- Processing input/output concurrently
- Summary
- Questions
- Further reading
- Introduction to Asynchronous Programming
- Technical requirements
- A quick analogy
- Asynchronous versus other programming models
- Asynchronous versus synchronous programming
- Asynchronous versus threading and multiprocessing
- An example in Python
- Summary
- Questions
- Further reading
- Implementing Asynchronous Programming in Python
- Technical requirements
- The asyncio module
- Coroutines event loops and futures
- Asyncio API
- The asyncio framework in action
- Asynchronously counting down
- A note about blocking functions
- Asynchronous prime-checking
- Improvements from Python 3.7
- Inherently blocking tasks
- concurrent.futures as a solution for blocking tasks
- Changes in the framework
- Examples in Python
- Summary
- Questions
- Further reading
- Building Communication Channels with asyncio
- Technical requirements
- The ecosystem of communication channels
- Communication protocol layers
- Asynchronous programming for communication channels
- Transports and protocols in asyncio
- The big picture of asyncio's server client
- Python example
- Starting a server
- Installing Telnet
- Simulating a connection channel
- Sending messages back to clients
- Closing the transports
- Client-side communication with aiohttp
- Installing aiohttp and aiofiles
- Fetching a website's HTML code
- Writing files asynchronously
- Summary
- Questions
- Further reading
- Deadlocks
- Technical requirements
- The concept of deadlock
- The Dining Philosophers problem
- Deadlock in a concurrent system
- Python simulation
- Approaches to deadlock situations
- Implementing ranking among resources
- Ignoring locks and sharing resources
- An additional note about locks
- Concluding note on deadlock solutions
- The concept of livelock
- Summary
- Questions
- Further reading
- Starvation
- Technical requirements
- The concept of starvation
- What is starvation?
- Scheduling
- Causes of starvation
- Starvation's relationship to deadlock
- The readers-writers problem
- Problem statement
- The first readers-writers problem
- The second readers-writers problem
- The third readers-writers problem
- Solutions to starvation
- Summary
- Questions
- Further reading
- Race Conditions
- Technical requirements
- The concept of race conditions
- Critical sections
- How race conditions occur
- Simulating race conditions in Python
- Locks as a solution to race conditions
- The effectiveness of locks
- Implementation in Python
- The downside of locks
- Turning a concurrent program sequential
- Locks do not lock anything
- Race conditions in real life
- Security
- Operating systems
- Networking
- Summary
- Questions
- Further reading
- The Global Interpreter Lock
- Technical requirements
- An introduction to the Global Interpreter Lock
- An analysis of memory management in Python
- The problem that the GIL addresses
- Problems raised by the GIL
- The potential removal of the GIL from Python
- How to work with the GIL
- Implementing multiprocessing rather than multithreading
- Getting around the GIL with native extensions
- Utilizing a different Python interpreter
- Summary
- Questions
- Further reading
- Designing Lock-Based and Mutex-Free Concurrent Data Structures
- Technical requirements
- Lock-based concurrent data structures in Python
- LocklessCounter and race conditions
- Embedding locks in the data structure of the counter
- The concept of scalability
- Analysis of the scalability of the counter data structure
- Approximate counters as a solution for scalability
- The idea behind approximate counters
- Implementing approximate counters in Python
- A few considerations for approximate counter designs
- Mutex-free concurrent data structures in Python
- The impossibility of being lock-free in Python
- Introduction to the network data structure
- Implementing a simple network data structure in Python and race conditions
- RCU as a solution
- Building on simple data structures
- Summary
- Questions
- Further reading
- Memory Models and Operations on Atomic Types
- Technical requirements
- Python memory model
- The components of Python memory manager
- Memory model as a labeled directed graph
- In the context of concurrency
- Atomic operations in Python
- What does it mean to be atomic?
- The GIL reconsidered
- Innate atomicity in Python
- Atomic versus nonatomic
- Simulation in Python
- Summary
- Questions
- Further reading
- Building a Server from Scratch
- Technical requirements
- Low-level network programming via the socket module
- The theory of server-side communication
- The API of the socket module
- Building a simple echo server
- Building a calculator server with the socket module
- The underlying calculation logic
- Implementing the calculator server
- Building a non-blocking server
- Analyzing the concurrency of the server
- Generators in Python
- Asynchronous generators and the send method
- Making the server non-blocking
- Summary
- Questions
- Further reading
- Testing Debugging and Scheduling Concurrent Applications
- Technical requirements
- Scheduling with APScheduler
- Installing APScheduler
- Not a scheduling service
- APScheduler functionalities
- APScheduler API
- Scheduler classes
- Executor classes
- Trigger keywords
- Common scheduler methods
- Examples in Python
- Blocking scheduler
- Background scheduler
- Executor pool
- Running on the cloud
- Testing and concurrency in Python
- Testing concurrent programs
- Unit testing
- Static code analysis
- Testing programs concurrently
- Debugging concurrent programs
- Debugging tools and techniques
- Debugging and concurrency
- Summary
- Questions
- Further reading
- Assessments
- Chapter 1
- Chapter 2
- Chapter 3
- Chapter 4
- Chapter 5
- Chapter 6
- Chapter 7
- Chapter 8
- Chapter 9
- Chapter 10
- Chapter 11
- Chapter 12
- Chapter 13
- Chapter 14
- Chapter 15
- Chapter 16
- Chapter 17
- Chapter 18
- Chapter 19
- Other Books You May Enjoy
- Leave a review - let other readers know what you think 更新時間：2021-06-10 19:25:01
