Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15382 Split one FsDatasetImpl lock to volume grain locks.
  3. HDFS-16429

Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.4.0
    • hdfs
    • Reviewed

    Description

      1、Use lockManager to maintain two level lock for FsDataSetImpl.

      The simple lock model like this.Parts of implemented as follows

      • As for finalizeReplica(),append(),createRbw()....First get BlockPoolLock read lock,and then get BlockPoolLock-volume-lock write lock.
      • As for getStoredBlock(),getMetaDataInputStream()....First get BlockPoolLock read lock,and the then get BlockPoolLock-volume-lock read lock.
      • As for deepCopyReplica(),getBlockReports() get the BlockPoolLock read lock.
      • As for delete hold the BlockPoolLock write lock.

      2、Make LightWeightResizableGSet become thread safe.It not become performance bottleneck if we make it thread safe.We can reduce lock grain size for ReplicaMap when make LightWeightResizableGSet thread safe.

      Attachments

        Issue Links

          Activity

            People

              Aiphag0 Mingxiang Li
              Aiphag0 Mingxiang Li
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 10m
                  2h 10m