Class DjlZoo

java.lang.Object
qupath.ext.djl.DjlZoo

public class DjlZoo extends Object
Helper class for working with DeepJavaLibrary model zoos.
  • Constructor Summary

    Constructors
    Constructor
    Description
     
  • Method Summary

    Modifier and Type
    Method
    Description
    static ai.djl.repository.zoo.Criteria<?,?>
    buildCriteria(ai.djl.repository.Artifact artifact, boolean allowDownload)
    Build Criteria from an existing artifact.
    static ai.djl.modality.Classifications
    classify(ai.djl.inference.Predictor<ai.djl.modality.cv.Image,ai.djl.modality.Classifications> predictor, BufferedImage img)
    Apply a classification model predictor to an input image.
    static qupath.lib.roi.interfaces.ROI
    createROI(ai.djl.modality.cv.output.BoundingBox box, qupath.lib.regions.ImageRegion region)
    Create a rectangular ROI from a bounding box, rescaling according to the ImageRegion.
    static qupath.lib.roi.interfaces.ROI
    createROI(ai.djl.modality.cv.output.DetectedObjects.DetectedObject obj, qupath.lib.regions.ImageRegion region)
    Create a ROI from a detected object, rescaling according to the ImageRegion.
    static qupath.lib.roi.interfaces.ROI
    createROI(ai.djl.modality.cv.output.Landmark landmark, qupath.lib.regions.ImageRegion region)
    Create a points ROI from a detected mask, rescaling according to the ImageRegion.
    static qupath.lib.roi.interfaces.ROI
    createROI(ai.djl.modality.cv.output.Mask mask, qupath.lib.regions.ImageRegion region, double threshold)
    Create a ROI from a detected mask, rescaling according to the ImageRegion.
    static ai.djl.modality.cv.output.DetectedObjects
    detect(ai.djl.inference.Predictor<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.DetectedObjects> predictor, BufferedImage img)
    Apply a detection model predictor to an input image.
    static Optional<List<qupath.lib.objects.PathObject>>
    detect(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.DetectedObjects> model, qupath.lib.images.ImageData<BufferedImage> imageData)
    Run object detection for an entire image.
    static Optional<List<qupath.lib.objects.PathObject>>
    detect(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.DetectedObjects> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects)
    Run object detection within specified objects in an image.
    imageToImage(ai.djl.inference.Predictor<ai.djl.modality.cv.Image,ai.djl.modality.cv.Image> predictor, BufferedImage img)
    Apply a predictor that takes a BufferedImage as input and provides another BufferedImage as output.
    static List<ai.djl.repository.MRL>
    List all zoo models for image classification.
    static List<ai.djl.repository.MRL>
    List all zoo models for instance segmentation.
    static List<ai.djl.repository.MRL>
    List all zoo models.
    static List<ai.djl.repository.MRL>
    listModels(ai.djl.Application application)
    List all zoo models for a specific application.
    static List<ai.djl.repository.MRL>
    listModels(ai.djl.repository.zoo.Criteria<?,?> criteria)
    List all zoo models matching a specific criteria.
    static List<ai.djl.repository.MRL>
    List all zoo models for object detection.
    static List<ai.djl.repository.MRL>
    List all zoo models for semantic segmentation.
    static ai.djl.repository.zoo.ZooModel<?,?>
    loadModel(ai.djl.repository.Artifact artifact, boolean allowDownload)
    Try to load a zoo model for a given artifact.
    static void
    Print all available zoo models to the log.
    static List<qupath.lib.objects.PathObject>
    segmentAnnotations(ai.djl.inference.Predictor<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> predictor, BufferedImage img, qupath.lib.regions.RegionRequest request)
    Apply a segmentation model to a single image, creating annotations from the result.
    static Optional<List<qupath.lib.objects.PathObject>>
    segmentAnnotations(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData)
    Apply a segmentation model to segment objects within an image, and represent these as annotations.
    static Optional<List<qupath.lib.objects.PathObject>>
    segmentAnnotations(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects)
    Apply a segmentation model to segment objects within specified objects in an image, and represent these as annotations.
    static List<qupath.lib.objects.PathObject>
    segmentDetections(ai.djl.inference.Predictor<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> predictor, BufferedImage img, qupath.lib.regions.RegionRequest request)
    Apply a segmentation model to a single image, creating detections from the result.
    static Optional<List<qupath.lib.objects.PathObject>>
    segmentDetections(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData)
    Apply a segmentation model to segment objects within an image, and represent these as detections.
    static Optional<List<qupath.lib.objects.PathObject>>
    segmentDetections(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects)
    Apply a segmentation model to segment objects within specified objects in an image, and represent these as detections.
    static List<qupath.lib.objects.PathObject>
    segmentObjects(ai.djl.inference.Predictor<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> predictor, BufferedImage img, qupath.lib.regions.RegionRequest request, qupath.lib.roi.interfaces.ROI roiMask, Function<qupath.lib.roi.interfaces.ROI,qupath.lib.objects.PathObject> creator, boolean skipBackground)
    Apply a segmentation model to a single image.
    static Optional<List<qupath.lib.objects.PathObject>>
    segmentObjects(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects, Function<qupath.lib.roi.interfaces.ROI,qupath.lib.objects.PathObject> creator, boolean skipBackground)
    Apply a segmentation model to segment objects within an image.
    toBufferedImage(ai.djl.modality.cv.Image image)
    Try to convert a DJL Image to a Java BufferedImage.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
  • Constructor Details

    • DjlZoo

      public DjlZoo()
  • Method Details

    • logAvailableModels

      public static void logAvailableModels()
      Print all available zoo models to the log.
    • listModels

      public static List<ai.djl.repository.MRL> listModels(ai.djl.repository.zoo.Criteria<?,?> criteria) throws ai.djl.repository.zoo.ModelNotFoundException, IOException
      List all zoo models matching a specific criteria.
      Parameters:
      criteria -
      Returns:
      Throws:
      ai.djl.repository.zoo.ModelNotFoundException
      IOException
    • listModels

      public static List<ai.djl.repository.MRL> listModels(ai.djl.Application application) throws ai.djl.repository.zoo.ModelNotFoundException, IOException
      List all zoo models for a specific application.
      Parameters:
      application -
      Returns:
      Throws:
      ai.djl.repository.zoo.ModelNotFoundException
      IOException
    • listModels

      public static List<ai.djl.repository.MRL> listModels() throws ai.djl.repository.zoo.ModelNotFoundException, IOException
      List all zoo models.
      Returns:
      Throws:
      ai.djl.repository.zoo.ModelNotFoundException
      IOException
    • listImageClassificationModels

      public static List<ai.djl.repository.MRL> listImageClassificationModels() throws ai.djl.repository.zoo.ModelNotFoundException, IOException
      List all zoo models for image classification.
      Returns:
      Throws:
      ai.djl.repository.zoo.ModelNotFoundException
      IOException
    • listSemanticSegmentationModels

      public static List<ai.djl.repository.MRL> listSemanticSegmentationModels() throws ai.djl.repository.zoo.ModelNotFoundException, IOException
      List all zoo models for semantic segmentation.
      Returns:
      Throws:
      ai.djl.repository.zoo.ModelNotFoundException
      IOException
    • listObjectDetectionModels

      public static List<ai.djl.repository.MRL> listObjectDetectionModels() throws ai.djl.repository.zoo.ModelNotFoundException, IOException
      List all zoo models for object detection.
      Returns:
      Throws:
      ai.djl.repository.zoo.ModelNotFoundException
      IOException
    • listInstanceSegmentationModels

      public static List<ai.djl.repository.MRL> listInstanceSegmentationModels() throws ai.djl.repository.zoo.ModelNotFoundException, IOException
      List all zoo models for instance segmentation.
      Returns:
      Throws:
      ai.djl.repository.zoo.ModelNotFoundException
      IOException
    • loadModel

      public static ai.djl.repository.zoo.ZooModel<?,?> loadModel(ai.djl.repository.Artifact artifact, boolean allowDownload) throws ai.djl.repository.zoo.ModelNotFoundException, ai.djl.MalformedModelException, IOException
      Try to load a zoo model for a given artifact.
      Parameters:
      artifact -
      allowDownload - optionally allow the model to be downloaded if it isn't currently available
      Returns:
      Throws:
      ai.djl.repository.zoo.ModelNotFoundException
      ai.djl.MalformedModelException
      IOException
    • buildCriteria

      public static ai.djl.repository.zoo.Criteria<?,?> buildCriteria(ai.djl.repository.Artifact artifact, boolean allowDownload) throws ai.djl.repository.zoo.ModelNotFoundException, ai.djl.MalformedModelException, IOException
      Build Criteria from an existing artifact. This can be used to then load a model.
      Parameters:
      artifact -
      allowDownload - optionally allow the model to be downloaded if it isn't currently available
      Returns:
      Throws:
      ai.djl.repository.zoo.ModelNotFoundException
      ai.djl.MalformedModelException
      IOException
    • createROI

      public static qupath.lib.roi.interfaces.ROI createROI(ai.djl.modality.cv.output.DetectedObjects.DetectedObject obj, qupath.lib.regions.ImageRegion region)
      Create a ROI from a detected object, rescaling according to the ImageRegion. Note that this adapts depending upon the object has a mask, landmark or bounding box. Any mask if thresholded at 0.5; if more control is needed, use createROI(Mask, ImageRegion, double).
      Parameters:
      obj -
      region -
      Returns:
    • createROI

      public static qupath.lib.roi.interfaces.ROI createROI(ai.djl.modality.cv.output.BoundingBox box, qupath.lib.regions.ImageRegion region)
      Create a rectangular ROI from a bounding box, rescaling according to the ImageRegion.
      Parameters:
      box -
      region -
      Returns:
    • createROI

      public static qupath.lib.roi.interfaces.ROI createROI(ai.djl.modality.cv.output.Mask mask, qupath.lib.regions.ImageRegion region, double threshold)
      Create a ROI from a detected mask, rescaling according to the ImageRegion.
      Parameters:
      mask -
      region -
      threshold -
      Returns:
    • createROI

      public static qupath.lib.roi.interfaces.ROI createROI(ai.djl.modality.cv.output.Landmark landmark, qupath.lib.regions.ImageRegion region)
      Create a points ROI from a detected mask, rescaling according to the ImageRegion.
      Parameters:
      landmark -
      region -
      Returns:
    • detect

      public static Optional<List<qupath.lib.objects.PathObject>> detect(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.DetectedObjects> model, qupath.lib.images.ImageData<BufferedImage> imageData) throws ai.djl.repository.zoo.ModelNotFoundException, ai.djl.MalformedModelException, IOException, ai.djl.translate.TranslateException
      Run object detection for an entire image.
      Parameters:
      model - the model
      imageData - the image within which to detect objects
      Returns:
      an optional containing a list of all objects added, or null if the detection was interrupted
      Throws:
      ai.djl.repository.zoo.ModelNotFoundException
      ai.djl.MalformedModelException
      IOException
      ai.djl.translate.TranslateException
    • detect

      public static Optional<List<qupath.lib.objects.PathObject>> detect(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.DetectedObjects> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects) throws ai.djl.repository.zoo.ModelNotFoundException, ai.djl.MalformedModelException, IOException, ai.djl.translate.TranslateException
      Run object detection within specified objects in an image.
      Parameters:
      model - the model
      imageData - the image within which to detect objects
      parentObjects - the parent objects, which become parents of what is detected; if null or the root object, the entire image is used for detection.
      Returns:
      an optional containing a list of all objects added, or null if the detection was interrupted
      Throws:
      ai.djl.repository.zoo.ModelNotFoundException
      ai.djl.MalformedModelException
      IOException
      ai.djl.translate.TranslateException
    • segmentAnnotations

      public static Optional<List<qupath.lib.objects.PathObject>> segmentAnnotations(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData) throws ai.djl.translate.TranslateException, IOException
      Apply a segmentation model to segment objects within an image, and represent these as annotations.
      Parameters:
      model - the segmentation model
      imageData - the image data
      Returns:
      an optional containing a list of all objects added, or null if the detection was interrupted
      Throws:
      ai.djl.translate.TranslateException
      IOException
      See Also:
    • segmentAnnotations

      public static Optional<List<qupath.lib.objects.PathObject>> segmentAnnotations(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects) throws ai.djl.translate.TranslateException, IOException
      Apply a segmentation model to segment objects within specified objects in an image, and represent these as annotations.
      Parameters:
      model - the segmentation model
      imageData - the image data
      parentObjects - parent objects within which the segmentation will be applied; if null or the root object, the full image will be used
      Returns:
      an optional containing a list of all objects added, or null if the detection was interrupted
      Throws:
      ai.djl.translate.TranslateException
      IOException
      See Also:
    • segmentDetections

      public static Optional<List<qupath.lib.objects.PathObject>> segmentDetections(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData) throws ai.djl.translate.TranslateException, IOException
      Apply a segmentation model to segment objects within an image, and represent these as detections.
      Parameters:
      model - the segmentation model
      imageData - the image data
      Returns:
      an optional containing a list of all objects added, or null if the detection was interrupted
      Throws:
      ai.djl.translate.TranslateException
      IOException
      See Also:
    • segmentDetections

      public static Optional<List<qupath.lib.objects.PathObject>> segmentDetections(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects) throws ai.djl.translate.TranslateException, IOException
      Apply a segmentation model to segment objects within specified objects in an image, and represent these as detections.
      Parameters:
      model - the segmentation model
      imageData - the image data
      parentObjects - parent objects within which the segmentation will be applied; if null or the root object, the full image will be used
      Returns:
      an optional containing a list of all objects added, or null if the detection was interrupted
      Throws:
      ai.djl.translate.TranslateException
      IOException
      See Also:
    • segmentObjects

      public static Optional<List<qupath.lib.objects.PathObject>> segmentObjects(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects, Function<qupath.lib.roi.interfaces.ROI,qupath.lib.objects.PathObject> creator, boolean skipBackground) throws ai.djl.translate.TranslateException, IOException
      Apply a segmentation model to segment objects within an image.
      Parameters:
      model - the segmentation model
      imageData - the image data
      parentObjects - parent objects within which the segmentation will be applied; if null or the root object, the full image will be used
      creator - function to create a new object; generally used to create annotations or detections
      skipBackground - if true, no object will be created for the background (label 0)
      Returns:
      an optional containing a list of all objects added, or null if the detection was interrupted
      Throws:
      ai.djl.translate.TranslateException
      IOException
    • segmentDetections

      public static List<qupath.lib.objects.PathObject> segmentDetections(ai.djl.inference.Predictor<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> predictor, BufferedImage img, qupath.lib.regions.RegionRequest request) throws ai.djl.translate.TranslateException
      Apply a segmentation model to a single image, creating detections from the result.
      Parameters:
      predictor - the segmentation model predictor
      img - the input image
      request - a region request; if not null, this is used to scale and translate the segmented ROIs
      Returns:
      a list of all objects that were segmented
      Throws:
      ai.djl.translate.TranslateException
      See Also:
    • segmentAnnotations

      public static List<qupath.lib.objects.PathObject> segmentAnnotations(ai.djl.inference.Predictor<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> predictor, BufferedImage img, qupath.lib.regions.RegionRequest request) throws ai.djl.translate.TranslateException
      Apply a segmentation model to a single image, creating annotations from the result.
      Parameters:
      predictor - the segmentation model predictor
      img - the input image
      request - a region request; if not null, this is used to scale and translate the segmented ROIs
      Returns:
      a list of all objects that were segmented
      Throws:
      ai.djl.translate.TranslateException
      See Also:
    • segmentObjects

      public static List<qupath.lib.objects.PathObject> segmentObjects(ai.djl.inference.Predictor<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.CategoryMask> predictor, BufferedImage img, qupath.lib.regions.RegionRequest request, qupath.lib.roi.interfaces.ROI roiMask, Function<qupath.lib.roi.interfaces.ROI,qupath.lib.objects.PathObject> creator, boolean skipBackground) throws ai.djl.translate.TranslateException
      Apply a segmentation model to a single image.
      Parameters:
      predictor - the segmentation model predictor
      img - the input image
      request - a region request; if not null, this is used to scale and translate the segmented ROIs
      roiMask - optional ROI to constrain the output; if not null, segmented ROIs will be intersected with the mask
      creator - function to convert a segmented ROI to an object (usually an annotation or detection)
      skipBackground - if true, no object will be created for the background (label 0)
      Returns:
      a list of all objects that were segmented
      Throws:
      ai.djl.translate.TranslateException
    • detect

      public static ai.djl.modality.cv.output.DetectedObjects detect(ai.djl.inference.Predictor<ai.djl.modality.cv.Image,ai.djl.modality.cv.output.DetectedObjects> predictor, BufferedImage img) throws ai.djl.translate.TranslateException
      Apply a detection model predictor to an input image.
      Parameters:
      predictor -
      img -
      Returns:
      Throws:
      ai.djl.translate.TranslateException
    • classify

      public static ai.djl.modality.Classifications classify(ai.djl.inference.Predictor<ai.djl.modality.cv.Image,ai.djl.modality.Classifications> predictor, BufferedImage img) throws ai.djl.translate.TranslateException
      Apply a classification model predictor to an input image.
      Parameters:
      predictor -
      img -
      Returns:
      Throws:
      ai.djl.translate.TranslateException
    • imageToImage

      public static BufferedImage imageToImage(ai.djl.inference.Predictor<ai.djl.modality.cv.Image,ai.djl.modality.cv.Image> predictor, BufferedImage img) throws ai.djl.translate.TranslateException
      Apply a predictor that takes a BufferedImage as input and provides another BufferedImage as output.
      Parameters:
      predictor -
      img -
      Returns:
      Throws:
      ai.djl.translate.TranslateException
    • toBufferedImage

      public static BufferedImage toBufferedImage(ai.djl.modality.cv.Image image) throws IllegalArgumentException
      Try to convert a DJL Image to a Java BufferedImage.
      Parameters:
      image -
      Returns:
      Throws:
      IllegalArgumentException - if the image is of an unknown type that cannot be converted