Package qupath.ext.djl
Class DjlZoo
java.lang.Object
qupath.ext.djl.DjlZoo
Helper class for working with DeepJavaLibrary model zoos.
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionstatic ai.djl.repository.zoo.Criteria<?, ?> buildCriteria(ai.djl.repository.Artifact artifact, boolean allowDownload) BuildCriteriafrom an existing artifact.static ai.djl.modality.Classificationsclassify(ai.djl.inference.Predictor<ai.djl.modality.cv.Image, ai.djl.modality.Classifications> predictor, BufferedImage img) Apply a classification model predictor to an input image.static qupath.lib.roi.interfaces.ROIcreateROI(ai.djl.modality.cv.output.BoundingBox box, qupath.lib.regions.ImageRegion region) Create a rectangular ROI from a bounding box, rescaling according to theImageRegion.static qupath.lib.roi.interfaces.ROIcreateROI(ai.djl.modality.cv.output.DetectedObjects.DetectedObject obj, qupath.lib.regions.ImageRegion region) Create a ROI from a detected object, rescaling according to theImageRegion.static qupath.lib.roi.interfaces.ROIcreateROI(ai.djl.modality.cv.output.Landmark landmark, qupath.lib.regions.ImageRegion region) Create a points ROI from a detected mask, rescaling according to theImageRegion.static qupath.lib.roi.interfaces.ROIcreateROI(ai.djl.modality.cv.output.Mask mask, qupath.lib.regions.ImageRegion region, double threshold) Create a ROI from a detected mask, rescaling according to theImageRegion.static ai.djl.modality.cv.output.DetectedObjectsdetect(ai.djl.inference.Predictor<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.DetectedObjects> predictor, BufferedImage img) Apply a detection model predictor to an input image.detect(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.DetectedObjects> model, qupath.lib.images.ImageData<BufferedImage> imageData) Run object detection for an entire image.detect(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.DetectedObjects> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects) Run object detection within specified objects in an image.static BufferedImageimageToImage(ai.djl.inference.Predictor<ai.djl.modality.cv.Image, ai.djl.modality.cv.Image> predictor, BufferedImage img) Apply a predictor that takes aBufferedImageas input and provides anotherBufferedImageas output.static List<ai.djl.repository.MRL> List all zoo models for image classification.static List<ai.djl.repository.MRL> List all zoo models for instance segmentation.static List<ai.djl.repository.MRL> List all zoo models.static List<ai.djl.repository.MRL> listModels(ai.djl.Application application) List all zoo models for a specific application.static List<ai.djl.repository.MRL> listModels(ai.djl.repository.zoo.Criteria<?, ?> criteria) List all zoo models matching a specific criteria.static List<ai.djl.repository.MRL> List all zoo models for object detection.static List<ai.djl.repository.MRL> List all zoo models for semantic segmentation.static ai.djl.repository.zoo.ZooModel<?, ?> loadModel(ai.djl.repository.Artifact artifact, boolean allowDownload) Try to load a zoo model for a given artifact.static voidPrint all available zoo models to the log.static List<qupath.lib.objects.PathObject> segmentAnnotations(ai.djl.inference.Predictor<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> predictor, BufferedImage img, qupath.lib.regions.RegionRequest request) Apply a segmentation model to a single image, creating annotations from the result.segmentAnnotations(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData) Apply a segmentation model to segment objects within an image, and represent these as annotations.segmentAnnotations(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects) Apply a segmentation model to segment objects within specified objects in an image, and represent these as annotations.static List<qupath.lib.objects.PathObject> segmentDetections(ai.djl.inference.Predictor<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> predictor, BufferedImage img, qupath.lib.regions.RegionRequest request) Apply a segmentation model to a single image, creating detections from the result.segmentDetections(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData) Apply a segmentation model to segment objects within an image, and represent these as detections.segmentDetections(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects) Apply a segmentation model to segment objects within specified objects in an image, and represent these as detections.static List<qupath.lib.objects.PathObject> segmentObjects(ai.djl.inference.Predictor<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> predictor, BufferedImage img, qupath.lib.regions.RegionRequest request, qupath.lib.roi.interfaces.ROI roiMask, Function<qupath.lib.roi.interfaces.ROI, qupath.lib.objects.PathObject> creator, boolean skipBackground) Apply a segmentation model to a single image.segmentObjects(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects, Function<qupath.lib.roi.interfaces.ROI, qupath.lib.objects.PathObject> creator, boolean skipBackground) Apply a segmentation model to segment objects within an image.static BufferedImagetoBufferedImage(ai.djl.modality.cv.Image image) Try to convert a DJLImageto a JavaBufferedImage.
-
Constructor Details
-
DjlZoo
public DjlZoo()
-
-
Method Details
-
logAvailableModels
public static void logAvailableModels()Print all available zoo models to the log. -
listModels
public static List<ai.djl.repository.MRL> listModels(ai.djl.repository.zoo.Criteria<?, ?> criteria) throws ai.djl.repository.zoo.ModelNotFoundException, IOExceptionList all zoo models matching a specific criteria.- Parameters:
criteria-- Returns:
- Throws:
ai.djl.repository.zoo.ModelNotFoundExceptionIOException
-
listModels
public static List<ai.djl.repository.MRL> listModels(ai.djl.Application application) throws ai.djl.repository.zoo.ModelNotFoundException, IOException List all zoo models for a specific application.- Parameters:
application-- Returns:
- Throws:
ai.djl.repository.zoo.ModelNotFoundExceptionIOException
-
listModels
public static List<ai.djl.repository.MRL> listModels() throws ai.djl.repository.zoo.ModelNotFoundException, IOExceptionList all zoo models.- Returns:
- Throws:
ai.djl.repository.zoo.ModelNotFoundExceptionIOException
-
listImageClassificationModels
public static List<ai.djl.repository.MRL> listImageClassificationModels() throws ai.djl.repository.zoo.ModelNotFoundException, IOExceptionList all zoo models for image classification.- Returns:
- Throws:
ai.djl.repository.zoo.ModelNotFoundExceptionIOException
-
listSemanticSegmentationModels
public static List<ai.djl.repository.MRL> listSemanticSegmentationModels() throws ai.djl.repository.zoo.ModelNotFoundException, IOExceptionList all zoo models for semantic segmentation.- Returns:
- Throws:
ai.djl.repository.zoo.ModelNotFoundExceptionIOException
-
listObjectDetectionModels
public static List<ai.djl.repository.MRL> listObjectDetectionModels() throws ai.djl.repository.zoo.ModelNotFoundException, IOExceptionList all zoo models for object detection.- Returns:
- Throws:
ai.djl.repository.zoo.ModelNotFoundExceptionIOException
-
listInstanceSegmentationModels
public static List<ai.djl.repository.MRL> listInstanceSegmentationModels() throws ai.djl.repository.zoo.ModelNotFoundException, IOExceptionList all zoo models for instance segmentation.- Returns:
- Throws:
ai.djl.repository.zoo.ModelNotFoundExceptionIOException
-
loadModel
public static ai.djl.repository.zoo.ZooModel<?,?> loadModel(ai.djl.repository.Artifact artifact, boolean allowDownload) throws ai.djl.repository.zoo.ModelNotFoundException, ai.djl.MalformedModelException, IOException Try to load a zoo model for a given artifact.- Parameters:
artifact-allowDownload- optionally allow the model to be downloaded if it isn't currently available- Returns:
- Throws:
ai.djl.repository.zoo.ModelNotFoundExceptionai.djl.MalformedModelExceptionIOException
-
buildCriteria
public static ai.djl.repository.zoo.Criteria<?,?> buildCriteria(ai.djl.repository.Artifact artifact, boolean allowDownload) throws ai.djl.repository.zoo.ModelNotFoundException, ai.djl.MalformedModelException, IOException BuildCriteriafrom an existing artifact. This can be used to then load a model.- Parameters:
artifact-allowDownload- optionally allow the model to be downloaded if it isn't currently available- Returns:
- Throws:
ai.djl.repository.zoo.ModelNotFoundExceptionai.djl.MalformedModelExceptionIOException
-
createROI
public static qupath.lib.roi.interfaces.ROI createROI(ai.djl.modality.cv.output.DetectedObjects.DetectedObject obj, qupath.lib.regions.ImageRegion region) Create a ROI from a detected object, rescaling according to theImageRegion. Note that this adapts depending upon the object has a mask, landmark or bounding box. Any mask if thresholded at 0.5; if more control is needed, usecreateROI(Mask, ImageRegion, double).- Parameters:
obj-region-- Returns:
-
createROI
public static qupath.lib.roi.interfaces.ROI createROI(ai.djl.modality.cv.output.BoundingBox box, qupath.lib.regions.ImageRegion region) Create a rectangular ROI from a bounding box, rescaling according to theImageRegion.- Parameters:
box-region-- Returns:
-
createROI
public static qupath.lib.roi.interfaces.ROI createROI(ai.djl.modality.cv.output.Mask mask, qupath.lib.regions.ImageRegion region, double threshold) Create a ROI from a detected mask, rescaling according to theImageRegion.- Parameters:
mask-region-threshold-- Returns:
-
createROI
public static qupath.lib.roi.interfaces.ROI createROI(ai.djl.modality.cv.output.Landmark landmark, qupath.lib.regions.ImageRegion region) Create a points ROI from a detected mask, rescaling according to theImageRegion.- Parameters:
landmark-region-- Returns:
-
detect
public static Optional<List<qupath.lib.objects.PathObject>> detect(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.DetectedObjects> model, qupath.lib.images.ImageData<BufferedImage> imageData) throws ai.djl.repository.zoo.ModelNotFoundException, ai.djl.MalformedModelException, IOException, ai.djl.translate.TranslateExceptionRun object detection for an entire image.- Parameters:
model- the modelimageData- the image within which to detect objects- Returns:
- an optional containing a list of all objects added, or null if the detection was interrupted
- Throws:
ai.djl.repository.zoo.ModelNotFoundExceptionai.djl.MalformedModelExceptionIOExceptionai.djl.translate.TranslateException
-
detect
public static Optional<List<qupath.lib.objects.PathObject>> detect(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.DetectedObjects> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects) throws ai.djl.repository.zoo.ModelNotFoundException, ai.djl.MalformedModelException, IOException, ai.djl.translate.TranslateExceptionRun object detection within specified objects in an image.- Parameters:
model- the modelimageData- the image within which to detect objectsparentObjects- the parent objects, which become parents of what is detected; if null or the root object, the entire image is used for detection.- Returns:
- an optional containing a list of all objects added, or null if the detection was interrupted
- Throws:
ai.djl.repository.zoo.ModelNotFoundExceptionai.djl.MalformedModelExceptionIOExceptionai.djl.translate.TranslateException
-
segmentAnnotations
public static Optional<List<qupath.lib.objects.PathObject>> segmentAnnotations(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData) throws ai.djl.translate.TranslateException, IOExceptionApply a segmentation model to segment objects within an image, and represent these as annotations.- Parameters:
model- the segmentation modelimageData- the image data- Returns:
- an optional containing a list of all objects added, or null if the detection was interrupted
- Throws:
ai.djl.translate.TranslateExceptionIOException- See Also:
-
segmentAnnotations
public static Optional<List<qupath.lib.objects.PathObject>> segmentAnnotations(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects) throws ai.djl.translate.TranslateException, IOExceptionApply a segmentation model to segment objects within specified objects in an image, and represent these as annotations.- Parameters:
model- the segmentation modelimageData- the image dataparentObjects- parent objects within which the segmentation will be applied; if null or the root object, the full image will be used- Returns:
- an optional containing a list of all objects added, or null if the detection was interrupted
- Throws:
ai.djl.translate.TranslateExceptionIOException- See Also:
-
segmentDetections
public static Optional<List<qupath.lib.objects.PathObject>> segmentDetections(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData) throws ai.djl.translate.TranslateException, IOExceptionApply a segmentation model to segment objects within an image, and represent these as detections.- Parameters:
model- the segmentation modelimageData- the image data- Returns:
- an optional containing a list of all objects added, or null if the detection was interrupted
- Throws:
ai.djl.translate.TranslateExceptionIOException- See Also:
-
segmentDetections
public static Optional<List<qupath.lib.objects.PathObject>> segmentDetections(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects) throws ai.djl.translate.TranslateException, IOExceptionApply a segmentation model to segment objects within specified objects in an image, and represent these as detections.- Parameters:
model- the segmentation modelimageData- the image dataparentObjects- parent objects within which the segmentation will be applied; if null or the root object, the full image will be used- Returns:
- an optional containing a list of all objects added, or null if the detection was interrupted
- Throws:
ai.djl.translate.TranslateExceptionIOException- See Also:
-
segmentObjects
public static Optional<List<qupath.lib.objects.PathObject>> segmentObjects(ai.djl.repository.zoo.ZooModel<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> model, qupath.lib.images.ImageData<BufferedImage> imageData, Collection<? extends qupath.lib.objects.PathObject> parentObjects, Function<qupath.lib.roi.interfaces.ROI, throws ai.djl.translate.TranslateException, IOExceptionqupath.lib.objects.PathObject> creator, boolean skipBackground) Apply a segmentation model to segment objects within an image.- Parameters:
model- the segmentation modelimageData- the image dataparentObjects- parent objects within which the segmentation will be applied; if null or the root object, the full image will be usedcreator- function to create a new object; generally used to create annotations or detectionsskipBackground- if true, no object will be created for the background (label 0)- Returns:
- an optional containing a list of all objects added, or null if the detection was interrupted
- Throws:
ai.djl.translate.TranslateExceptionIOException
-
segmentDetections
public static List<qupath.lib.objects.PathObject> segmentDetections(ai.djl.inference.Predictor<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> predictor, BufferedImage img, qupath.lib.regions.RegionRequest request) throws ai.djl.translate.TranslateExceptionApply a segmentation model to a single image, creating detections from the result.- Parameters:
predictor- the segmentation model predictorimg- the input imagerequest- a region request; if not null, this is used to scale and translate the segmented ROIs- Returns:
- a list of all objects that were segmented
- Throws:
ai.djl.translate.TranslateException- See Also:
-
segmentAnnotations
public static List<qupath.lib.objects.PathObject> segmentAnnotations(ai.djl.inference.Predictor<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> predictor, BufferedImage img, qupath.lib.regions.RegionRequest request) throws ai.djl.translate.TranslateExceptionApply a segmentation model to a single image, creating annotations from the result.- Parameters:
predictor- the segmentation model predictorimg- the input imagerequest- a region request; if not null, this is used to scale and translate the segmented ROIs- Returns:
- a list of all objects that were segmented
- Throws:
ai.djl.translate.TranslateException- See Also:
-
segmentObjects
public static List<qupath.lib.objects.PathObject> segmentObjects(ai.djl.inference.Predictor<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.CategoryMask> predictor, BufferedImage img, qupath.lib.regions.RegionRequest request, qupath.lib.roi.interfaces.ROI roiMask, Function<qupath.lib.roi.interfaces.ROI, throws ai.djl.translate.TranslateExceptionqupath.lib.objects.PathObject> creator, boolean skipBackground) Apply a segmentation model to a single image.- Parameters:
predictor- the segmentation model predictorimg- the input imagerequest- a region request; if not null, this is used to scale and translate the segmented ROIsroiMask- optional ROI to constrain the output; if not null, segmented ROIs will be intersected with the maskcreator- function to convert a segmented ROI to an object (usually an annotation or detection)skipBackground- if true, no object will be created for the background (label 0)- Returns:
- a list of all objects that were segmented
- Throws:
ai.djl.translate.TranslateException
-
detect
public static ai.djl.modality.cv.output.DetectedObjects detect(ai.djl.inference.Predictor<ai.djl.modality.cv.Image, ai.djl.modality.cv.output.DetectedObjects> predictor, BufferedImage img) throws ai.djl.translate.TranslateExceptionApply a detection model predictor to an input image.- Parameters:
predictor-img-- Returns:
- Throws:
ai.djl.translate.TranslateException
-
classify
public static ai.djl.modality.Classifications classify(ai.djl.inference.Predictor<ai.djl.modality.cv.Image, ai.djl.modality.Classifications> predictor, BufferedImage img) throws ai.djl.translate.TranslateExceptionApply a classification model predictor to an input image.- Parameters:
predictor-img-- Returns:
- Throws:
ai.djl.translate.TranslateException
-
imageToImage
public static BufferedImage imageToImage(ai.djl.inference.Predictor<ai.djl.modality.cv.Image, ai.djl.modality.cv.Image> predictor, BufferedImage img) throws ai.djl.translate.TranslateExceptionApply a predictor that takes aBufferedImageas input and provides anotherBufferedImageas output.- Parameters:
predictor-img-- Returns:
- Throws:
ai.djl.translate.TranslateException
-
toBufferedImage
public static BufferedImage toBufferedImage(ai.djl.modality.cv.Image image) throws IllegalArgumentException Try to convert a DJLImageto a JavaBufferedImage.- Parameters:
image-- Returns:
- Throws:
IllegalArgumentException- if the image is of an unknown type that cannot be converted
-