UNIX系统内幕PDF|Epub|txt|kindle电子书版本网盘下载

UNIX系统内幕PDF格式电子书版下载

注意：本站所有压缩包均有解压码： 点击下载压缩包解压工具

1 INTRODUCTION1

1.1 Introduction1

1.1.2 The Beginning2

1.1.1 A Brief History2

1.1.3 Proliferation3

1.1.4 BSD4

1.1.6 Commercialization5

1.1.5 System Ⅴ5

1.1.8 Standards6

1.1.7 Mach6

1.1.9 OSF and UI7

1.2 The Mandate for Change8

1.1.10 SVR4 and Beyond8

1.2.2 Networking9

1.2.1 Functionality9

1.2.4 Hardware Changes10

1.2.3 Performance10

1.2.6 Paradigm Shifts11

1.2.5 Quality Improvement11

1.2.8 Smallis Beautiful12

1.2.7 Other Application Domains12

1.2.9 Flexibility13

1.3.1 What was Good about UNIX?14

1.3 Looking Back,Looking Forward14

1.3.2 What is Wrong with UNIX?15

1.4 The Scope of this Book16

1.5 References17

2.1 Introduction19

2 THE PROCESS AND THE KERNEL19

2.2 Mode,Space,and Context22

2.3 The Process Abstraction24

2.3.1 Process State25

2.3.2 Process Context26

2.3.3 User Credentials27

2.3.4 The u Area and the proc Structure28

2.4 Executing in Kernel Mode30

2.4.2 Interrupt Handling31

2.4.1 The System Call Interface31

2.5 Synchronization33

2.5.2 Interrupts35

2.5.1 Blocking Operations35

2.6 Process Scheduling37

2.5.3 Multiprocessors37

2.7 Signals38

2.8.1 forkand exec39

2.8 New Processes and Programs39

2.8.3 fork Optimization41

2.8.2 Process Creation41

2.8.4 Invoking aNew Program42

2.8.5 Process Termination43

2.8.6 Awaiting Process Termination44

2.10 Exercises45

2.9 Summary45

2.8.7 Zombie Processes45

2.11 References46

3.1 Introduction48

3 THREADS AND LIGHTWEIGHT PROCESSES48

3.1.2 Multiple Threads and Processors49

3.1.1 Motivation49

3.2 Fundamental Abstractions52

3.1.3 Concurrency and Parallelism52

3.2.2 Lightweight Processes53

3.2.1 Kernel Threads53

3.2.3 User Threads55

3.3.1 Semantics offork58

3.3 Lightweight Process Design—Issues to Consider58

3.3.2 Other System Calls59

3.3.3 Signal Delivery and Handling60

3.3.5 Stack Growth61

3.3.4 Visibility61

3.4.2 Implementing Threads Libraries62

3.4.1 The Programming Interface62

3.4 User-Level Threads Libraries62

3.5 Scheduler Activations64

3.6.1 Kernel Threads65

3.6 Multithreading in Solaris and SVR465

3.6.2 Lightweight Process Implementation66

3.6.3 User Threads67

3.6.5 Interrupt Handling68

3.6.4 User Thread Implementation68

3.7.1 The Mach Abstractions—Tasks and Threads70

3.7 Threads in Mach70

3.6.6 System Call Handling70

3.7.2 Mach C-threads71

3.8.1 The UNIX Interface72

3.8 Digital UNIX72

3.8.2 System Calls and Signals74

3.8.3 The pthreads Library75

3.9.1 Programming Models76

3.9 Mach 3.0 Continuations76

3.9.2 Using Continuations77

3.9.3 Optimizations78

3.10 Summary79

3.9.4 Analysis79

3.12 References80

3.11 Exercises80

4.1 Introduction83

4 SIGNALS AND SESSION MANAGEMENT83

4.2.1 Signal Handling84

4.2 Signal Generation and Handling84

4.2.3 Typical Scenarios87

4.2.2 Signal Generation87

4.2.4 Sleep and Signals88

4.3 Unreliable Signals89

4.4.1 Primary Features90

4.4 Reliable Signals90

4.4.2 The SVR3 Implementation91

4.4.3 BSD Signal Management92

4.5 Signals in SVR493

4.6 Signals Implementation94

4.7 Exceptions95

4.6.2 Delivery and Handling95

4.6.1 Signal Generation95

4.8 Mach Exception Handling96

4.8.1 Exception Ports97

4.8.3 Debugger Interactions98

4.8.2 Error Handling98

4.9.1 Common Concepts99

4.9 Process Groups and Terminal Management99

4.8.4 Analysis99

4.9.2 The SVR3 Model100

4.9.3 Limitations102

4.9.4 4.3BSD Groups and Terminals103

4.9.5 Drawbacks104

4.10.1 Motivation105

4.10 The SVR4 Sessions Architecture105

4.10.2 Sessions and Process Groups106

4.10.4 Controlling Terminals107

4.10.3 Data Structures107

4.10.5 The 4.4BSD Sessions Implementation109

4.12 Exercises110

4.11 Summary110

4.13 References111

5.1 Introduction112

5 PROCESS SCHEDULING112

5.2 Clock Interrupt Handling113

5.2.1 Caliouts114

5.2.2 Alarms115

5.3 Scheduler Goals116

5.4 Traditional UNIX Scheduling117

5.4.1 Process Priorities118

5.4.2 Scheduler Implementation119

5.4.3 Run Queue Manipulation120

5.4.4 Analysis121

5.5.1 The Class-Independent Layer122

5.5 The SVR4 Scheduler122

5.5.2 Interface to the Scheduling Classes124

5.5.3 The Time-Sharing Class126

5.5.4 The Real-Time Class127

5.5.6 Analysis129

5.5.5 The priocntl System Call129

5.6 Solaris 2.x Scheduling Enhancements130

5.6.2 Multiprocessor Support131

5.6.1 Preemptive Kernel131

5.6.4 Priority Inversion133

5.6.3 Hidden Scheduling133

5.6.5 Implementation of Priority Inheritance135

5.6.6 Limitations of Priority Inheritance137

5.6.7 Turnstiles138

5.7 Scheduling in Mach139

5.6.8 Analysis139

5.7.1 Multiprocessor Support140

5.8 The Digital UNIX Real-Time Scheduler142

5.9 Other Scheduling Implementations143

5.8.1 Multiprocessor Support143

5.9.2 Deadline-Driven Scheduling144

5.9.1 Fair-Share Scheduling144

5.9.3 A Three-Level Scheduler145

5.11 Exercises146

5.10 Summary146

5.12 References147

6.1 Introduction149

6 INTERPROCESS COMMUNICATIONS149

6.2.1 Signals150

6.2 Universal IPC Facilities150

6.2.2 Pipes151

6.2.3 SVR4Pipes152

6.2.4 Process Tracing153

6.3.1 Common Elements155

6.3 System Ⅴ IPC155

6.3.2 Semaphores156

6.3.3 Message Queues160

6.3.4 Shared Memory162

6.3.5 Discussion164

6.4 Mach IPC165

6.4.1 Basic Concepts166

6.5.1 Message Data Structures167

6.5 Messages167

6.5.2 Message Passing Interface169

6.6.2 The Port Data Structure170

6.6.1 The Port Name Space170

6.6 Ports170

6.6.3 Port Translations171

6.7 Message Passing172

6.7.1 Transferring Port Rights173

6.7.2 Out-of-Line Memory175

6.8 Port Operations177

6.7.4 Notifications177

6.7.3 Control Flow177

6.8.2 Backup Ports178

6.8.1 Destroying a Port178

6.8.3 Port Sets179

6.8.4 Port Interpolation180

6.9 Extensibility181

6.10 Mach 3.0 Enhancements182

6.10.3 User-Reference Counting of Send Rights183

6.10.2 Mach 3.0 Notifications183

6.10.1 Send-Once Rights183

6.11 Discussion184

6.13 Exercises185

6.12 Summary185

6.14 References186

7.1 Introduction187

7 SYNCHRONIZATION AND MULTIPROCESSING187

7.2 Synchronization in Traditional UNIX Kernels188

7.2.2 Sleep and Wakeup189

7.2.1 Interrupt Masking189

7.2.3 Limitations of Traditional Approach190

7.3.1 Memory Model191

7.3 Multiprocessor Systems191

7.3.2 Synchronization Support193

7.4 Multiprocessor Synchronization Issues195

7.3.3 Software Architecture195

7.4.2 The Thundering Herd Problem196

7.4.1 The Lost Wakeup Problem196

7.5 Semaphores197

7.5.2 Event-Wait Using Semaphores198

7.5.1 Semaphores to Provide Mutual Exclusion198

7.5.4 Drawbacks of Semaphores199

7.5.3 Semaphores to Control Countable Resources199

7.5.5 Convoys200

7.6 Spin Locks201

7.6.1 Use of Spin Locks202

7.7 Condition Variables203

7.7.1 Implementation Issues204

7.7.3 Blocking Locks205

7.7.2 Events205

7.8.1 Design Considerations206

7.8 Read-Write Locks206

7.8.2 Implementation207

7.10.1 Deadlock Avoidance209

7.10 Other Considerations209

7.9 Reference Counts209

7.10.3 To Block or to Spin211

7.10.2 Recursive Locks211

7.10.5 Granularity and Duration212

7.10.4 What to Lock212

7.11.1 SVR4.2/MP213

7.11 Case Studies213

7.11.2 Digitel UNIX214

7.11.3 Other Implementations216

7.13 Exercises217

7.12 Summary217

7.14 References218

8.1 Introduction220

8 FILE SYSTEM INTERFACE AND FRAMEWORK220

8.2.1 Files and Directories221

8.2 The User Interface to Files221

8.2.2 File Attributes223

8.2.3 File Descriptors225

8.2.4 File I/O227

8.2.6 File Locking228

8.2.5 Scatter-GatherI/O228

8.3 File Systems229

8.3.1 Logical Disks230

8.4.1 Symbolic Links231

8.4 Special Files231

8.5 File System Framework233

8.4.2 Pipes and FIFOs233

8.6.1 Objectives234

8.6 The Vnode/Vfs Architecture234

8.6.2 Lessons from DeviceI/O235

8.6.3 Overview ofthe Vnode/Vfs Interface238

8.7.2 Vnodes and Open Files240

8.7.1 Objectives240

8.7 Implementation Overview240

8.7.3 The Vnode241

8.7.4 Vnode Reference Count242

8.7.5 The Vfs Object243

8.8.1 The Per-File Private Data244

8.8 File-System-Dependent Objects244

8.8.2 The vnodeops Vector245

8.8.3 File-System-Dependent Parts of the Vfs Layer246

8.9.1 The Virtual File System Switch247

8.9 Mounting a File System247

8.9.2 mount Implementation248

8.10.1 Pathname Traversal249

8.10 Operations on Files249

8.9.3 VFS_MOUNT Processing249

8.10.2 Directory Lookup Cache250

8.10.3 The VOP_LOOKUP Operation251

8.10.4 Opening a File252

8.10.7 User Credentials253

8.10.6 File Attributes253

8.10.5 File I/O253

8.11 Analysis254

8.11.1 Drawbacks ofthe SVR4 Implementation255

8.11.2 The 4.4BSD Model256

8.12 Summary257

8.11.3 The OSF/1 Approach257

8.13 Exercises258

8.14 References259

9.1 Introduction261

9 FILE SYSTEM IMPLEMENTATIONS261

9.2 The System Ⅴ File System(s5fs)262

9.2.2 Inodes263

9.2.1 Directories263

9.2.3 The Superblock266

9.3.2 Inode Lookup267

9.3.1 In-Core Inodes267

9.3 S5fs Kernel Organization267

9.3.3 File I/O268

9.3.4 Allocating and Reclaiming Inodes270

9.4 Analysis of s5fs271

9.6 Hard Disk Structure272

9.5 The Berkeley Fast File System272

9.7.1 Blocks and Fragments273

9.7 On-Disk Organization273

9.7.2 Allocation Policies274

9.8 FFS Functionality Enhancements275

9.9 Analysis276

9.10.1 The Memory File System278

9.10 Temporary File Systems278

9.10.2 The tmpfs File System279

9.11.1 The Specfs File System280

9.11 Special-Purpose File Systems280

9.11.2 The/proc File System281

9.11.4 The Translucent File System283

9.11.3 The Processor File System283

9.12 The Old Buffer Cache284

9.12.1 Basic Operation285

9.12.3 Advantages286

9.12.2 Buffer Headers286

9.12.5 Ensuring File System Consistency287

9.12.4 Disadvantages287

9.14 Exercises288

9.13 Summary288

9.15 References289

10.1 Introduction291

10 DISTRIBUTED FILE SYSTEMS291

10.2.1 Design Considerations292

10.2 GeneraI Characteristics of Distributed File Systems292

10.3 Network File System(NFS)293

10.3.2 Design Goals294

10.3.1 User Perspective294

10.3.3 NFS Components295

10.3.4 Statelessness297

10.4.1 Extended Data Representation(XDR)298

10.4 The Protocol Suite298

10.4.2 Remote Procedure Calls(RPC)300

10.5 NFS Implementation301

10.5.2 File Handles302

10.5.1 Control Flow302

10.5.4 Pathname Lookup303

10.5.3 The Mount Operation303

10.6.1 Open File Permissions304

10.6 UNIX Semantics304

10.6.3 Reads and Writes305

10.6.2 Deletion of Open Files305

10.7.2 Client-Side Caching306

10.7.1 Performance Bottlenecks306

10.7 NFS Performance306

10.7.3 Deferral ofWrites307

10.7.4 The Retransmissions Cache308

10.8.1 The Auspex Functional Multiprocessor Architecture309

10.8 Dedicated NFS Servers309

10.8.2 IBM's HA-NFS Server310

10.9.1 NFS Access Control312

10.9 NFS Security312

10.9.3 Root Remapping313

10.9.2 UID Remapping313

10.10 NFS Version 3314

10.12 RFS Architecture315

10.11 Remote File Sharing(RFS)315

10.12.1 Remote Message Protocol316

10.13.1 Remote Mount317

10.13 RFS Implementation317

10.12.2 Stateful Operation317

10.13.2 RFS Clients and Servers319

10.13.3 Crash Recovery320

10.14 Client-Side Caching321

10.13.4 Other Issues321

10.14.1 Cache Consistency322

10.15.1 Scalable Architecture323

10.15 The Andrew File System323

10.15.2 Storage and Name Space Organization324

10.15.3 Session Semantics325

10.16.1 Caching and Consistency326

10.16 AFS Implementation326

10.16.3 Security327

10.16.2 Pathname Lookup327

10.17 AFS Shortcomings328

10.18.1 DFS Architecture329

10.18 The DCE Distributed File System(DCE DFS)329

10.18.2 Cache Consistency330

10.18.4 Other DFS Services332

10.18.3 The Token Manager332

10.18.5 Analysis333

10.20 Exercises334

10.19 Summary334

10.21 References335

11.1 Introduction338

11 ADVANCED FILE SYSTEMS338

11.2 Limitations of Traditional File Systems339

11.2.1 FFS Disk Layout340

11.2.2 Predominance of Writes341

11.2.4 Crash Recovery342

11.2.3 Metadata Updates342

11.3 File System Clustering(Sun-FFS)343

11.4.1 Basic Characteristics344

11.4 The Journaling Approach344

11.5 Log-Structured File Systems345

11.6 The 4.4BSD Log-Structured File System346

11.6.2 Data Retrieval347

11.6.1 Writing the Log347

11.6.3 Crash Recovery348

11.6.5 Analysis349

11.6.4 The Cleaner Process349

11.7 Metadata Logging350

11.7.1 Normal Operation351

11.7.2 Log Consistency352

11.7.3 Recovery353

11.7.4 Analysis354

11.8 The Episode File System355

11.8.1 Basic Abstractions356

11.8.2 Structure357

11.8.4 Other Features358

11.8.3 Logging358

11.9 Watchdogs359

11.9.2 Message Channels360

11.9.1 Directory Watchdogs360

11.9.3 Applications361

11.10 The 4.4BSD Portal File System362

11.10.1 Using Portals363

11.11.1 Framework and Interface364

11.11 Stackable File System Layers364

11.11.2 The SunSoft Prototype366

11.12 The 4.4BSD File System Interface367

11.14 Exercises368

11.13 Summary368

11.12.1 The Nullfs and Union Mount File Systems368

11.15 References369

12.1 Introduction372

12 KERNEL MEMORY ALLOCATION372

12.2.1 Evaluation Criteria374

12.2 Functional Requirements374

12.3 Resource Map Allocator376

12.3.1 Analysis377

12.4 Simple Power-of-Two Free Lists379

12.4.1 Analysis380

12.5 The McKusick-Karels Allocator381

12.6 The Buddy System383

12.5.1 Analysis383

12.6.1 Analysis385

12.7.1 Lazy Coalescing386

12.7 The SVR4 Lazy Buddy Algorithm386

12.7.2 SVR4 Implementation Details387

12.8.1 Garbage Collection388

12.8 The Mach-OSF/1 Zone Allocator388

12.8.2 Analysis389

12.9 A Hierarchical Allocator for Multiprocessors390

12.10.1 Object Reuse392

12.10 The Solaris 2.4 Slab Allocator392

12.9.1 Analysis392

12.10.2 Hardware Cache Utilization393

12.10.4 Design and Interfaces394

12.10.3 Allocator Footprint394

12.10.5 Implementation395

12.10.6 Analysis396

12.11 Summary397

12.12 Exercises398

12.13 References399

13.1 Introduction400

13 VIRTUAL MEMORY400

13.1.1 Memory Management in the Stone Age401

13.2.1 Functional Requirements404

13.2 Demand Paging404

13.2.2 The Virtual Address Space406

13.2.4 The Swap Area407

13.2.3 Initial Access to a Page407

13.2.5 Translation Maps408

13.2.6 Page Replacement Policies409

13.3 Hardware Requirements410

13.3.1 MMU Caches412

13.3.2 The Intel 80x86413

13.3.3 The IBM RS/6000416

13.3.4 The MIPS R3000419

13.4.1 Physical Memory421

13.4 4.3BSD—A Case Study421

13.4.2 The Address Space423

13.4.3 Where IsthePage?424

13.4.4 Swap Space426

13.5.1 Process Creation427

13.5 4.3BSD Memory Management Operations427

13.5.2 Page Fault Handling428

13.5.3 The Free Page List431

13.5.4 Swapping432

13.6 Analysis433

13.7 Exercises435

13.8 References436

14.1 Motivation437

14 THE SVR4 VM ARCHITECTURE437

14.2 Memory-Mapped Files438

14.3 VM Design Principles440

14.2.1 mmap and Related System Calls440

14.4 Fundamental Abstractions441

14.4.1 Physical Memory442

14.4.2 The Address Space443

14.4.3 Address Mappings444

14.4.4 Anonymous Pages445

14.4.5 Hardware Address Translation446

14.5.1 seg_vn448

14.5 Segment Drivers448

14.5.2 seg_map449

14.5.5 seg_kp450

14.5.4 seg_kmem450

14.5.3 seg_dev450

14.6 The Swap Layer451

14.7 VM Operations452

14.7.2 Anonymous Page Handling453

14.7.1 Creating a New Mapping453

14.7.4 Sharing Anonymous Pages455

14.7.3 Process Creation455

14.7.5 Page Fault Handling457

14.7.6 Shared Memory458

14.7.7 Other Components459

14.8.1 Vnode Interface Changes460

14.8 Interaction with the Vnode Subsystem460

14.8.2 Unifying File Access461

14.8.3 Miscellaneous Issues463

14.9.2 Virtual Swap Management464

14.9.1 Extended Swap Space464

14.9 Virtual Swap Space in Solaris464

14.10 Analysis466

14.9.3 Discussion466

14.11.1 Causes of High Fault Rates468

14.11 Performance Improvements468

14.11.2 SVR4 Enhancements to the SunOS VM Implementation469

14.12 Summary470

14.11.3 Results and Discussion470

14.14 References471

14.13 Exercises471

15.2 Mach Memory Management Design473

15.1 Introduction473

15 MORE MEMORY MANAGEMENT TOPICS473

15.2.2 Programming Interface474

15.2.1 Design Goals474

15.2.3 Fundamental Abstractions476

15.3.1 Copy-on-Write Sharing478

15.3 Memory Sharing Facilities478

15.3.2 Read-Write Sharing480

15.4.1 Memory Object Initialization481

15.4 Memory Objects and Pagers481

15.4.2 Interface between the Kernel and the Pager482

15.5 External and Internal Pagers484

15.4.3 Kernel-Pager Interactions484

15.5.1 A Network Shared Memory Server485

15.6 Page Replacement487

15.7 Analysis489

15.8 Memory Management in 4.4BSD490

15.9 Translation Lookaside Buffer(TLB)Consistency492

15.9.1 TLB Consistency on a Uniprocessor493

15.10 TLB Shootdown in Mach494

15.9.2 Multiprocessor Issues494

15.10.1 Synchronization and Deadlock Avoidance495

15.10.2 Discussion496

15.11.1 SVR4/MP497

15.11 TLB Consistency in SVR4 and SVR4.2 UNIX497

15.11.2 SVR4.2/MP498

15.11.3 Lazy Shootdowns499

15.11.5 Discussion500

15.11.4 Immediate Shootdowns500

15.12 Other TLB Consistency Algorithms501

15.13 Virtually Addressed Caches502

15.13.1 Mapping Changes504

15.13.3 DMA Operations505

15.13.2 Address Aliases505

15.13.4 Maintaining Cache Consistency506

15.14 Exercises507

15.13.5 Analysis507

15.15 References508

16.2 Overview511

16.1 Introduction511

16 DEVICE DRIVERS AND I/O511

16.2.1 Hardware Configuration513

16.2.2 Device Interrupts514

16.3 Device Driver Framework516

16.3.1 Classifying Devices and Drivers516

16.3.2 Invoking Driver Code517

16.3.3 The Device Switches518

16.3.4 Driver Entry Points519

16.4 The I/O Subsystem520

16.4.1 Major and Minor Device Numbers521

16.4.2 Device Files522

16.4.3 The specfs File System523

16.4.4 The Common snode524

16.4.5 Device Cloning526

16.4.6 I/O to a Character Device526

16.5 The poll System Call527

16.5.1 poll Implementation528

16.5.2 The 4.3BSD select System Call529

16.6 Block I/O530

16.6.1 The buf Structure531

16.6.2 Interaction with the Vnode532

16.6.3 Device Access Methods533

16.6.4 Raw I/O to a Block Device535

16.7 The DDI/DKI Specification535

16.7.1 General Recommendations537

16.7.2 Section 3 Functions537

16.7.3 Other Sections538

16.8 Newer SVR4 Releases539

16.8.1 Multiprocessor-Safe Drivers540

16.8.2 SVR4.1/ES Changes540

16.8.3 Dynamic Loading and Unloading541

16.9 Future Directions543

16.10 Summary544

16.11 Exercises545

16.12 References545

17 STREAMS547

17.1 Motivation547

17.2 Overview548

17.3 Messages and Queues551

17.3.1 Messages551

17.3.2 Virtual Copying552

17.3.3 Message Types553

17.3.4 Queues and Modules554

17.4 Stream I/O556

17.4.1 The STREAMS Scheduler557

17.4.2 Priority Bands558

17.4.3 Flow Control558

17.4.4 The Driver End560

17.4.5 The Stream Head561

17.5 Configuration and Setup562

17.5.1 Configuring a Module or Driver562

17.5.2 Opening a Stream564

17.5.3 Pushing Modules565

17.5.4 Clone Devices566

17.6 STREAMS ioctls566

17.6.1 I_STR ioctl Processing567

17.6.2 Transparent ioctls568

17.7 Memory Allocation568

17.7.1 Extended STREAMS Buffers570

17.8 Multiplexing571

17.8.1 Upper Multiplexors571

17.8.2 Lower Multiplexors572

17.8.3 Linking Streams572

17.8.4 Data Flow574

17.8.5 Ordinary and Persistent Links575

17.9 FIFOs and Pipes576

17.9.1 STREAMS FIFOs576

17.9.2 STREAMS Pipes577

17.10 Networking Interfaces578

17.10.1 Transport Provider Interface(TPI)579

17.10.2 Transport Layer Interface(TLI)579

17.10.3 Sockets580

17.10.4 SVR4 Sockets Implementation582

17.11 Summary583

17.12 Exercises584

17.13 References585

Index587