This document describes support for capturing 3D depth photo, enhancing photo processing by leveraging the 3D depth information and I/O with [[!XDM]] format.
This document was published by the Crosswalk Project as an API Draft. If you wish to make comments regarding this document, please send them to crosswalk-dev@lists.crosswalk-project.org. All comments are welcome.
The APIs described in this document are exposed through
realsense.DepthEnabledPhotography module.
DepthMask
The DepthMask interface provides methods for
depth-enabled depth mask generator algorithm.
If failed to create, an InitFailureException will be thrown.
The init() method initializes the Depth Mask function
with the photo that needs processing.
This method returns a promise.
The promise will be fulfilled if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The computeFromCoordinate() method creates a mask directly from
a depth coordinate.
The mask image is in DEPTH_F32 pixel format.
This method returns a promise.
The promise will be fulfilled with the mask image if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The computeFromThreshold() method creates a mask from depthThreshold.
The mask image is in DEPTH_F32 pixel format.
This method returns a promise.
The promise will be fulfilled with the mask image if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
DepthRefocus
The DepthRefocus interface provides methods for
depth-enabled refocus algorithm.
If failed to create, an InitFailureException will be thrown.
The init() method initializes the Depth Refocus function
with the photo that needs processing.
This method returns a promise.
The promise will be fulfilled if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The apply() method refocuses the image at input focusPoint
by using depth data to refocus.
This method returns a promise.
The promise will be fulfilled with the refocused photo instance if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
Image
The image buffer.
The image pixel format.
The image width in pixels.
The image height in pixels.
The image buffer.
Measurement
The Measurement interface provides methods for
depth-enabled measurement algorithm.
If failed to create, an InitFailureException will be thrown.
The measureDistance() method measures the distance
between the starting and ending points in mm.
This method returns a promise.
The promise will be fulfilled with the measure data if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
measureUADistance() method measures the distance
between the starting and ending points in mm by using an experimental algorithm for a User Assisted(UA) measure.
This method returns a promise.
The promise will be fulfilled with the measure data if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
queryUADataSize() method returns the size of the MeasureData possibilities.
This method returns a promise.
The promise will be fulfilled with the size of the MeasureData if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
queryUAData() method returns an array of the MeasureData possibilites.
The size of the array is equal to the value returned by the queryUADataSize().
This method returns a promise.
The promise will be fulfilled with an array of the MeasureData if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
MotionEffect
The MotionEffect interface provides methods for
6DoF parallax feature.
If failed to create, an InitFailureException will be thrown.
The init() method initializes the 6DoF parallax function
with the photo that needs processing.
This method returns a promise.
The promise will be fulfilled if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The apply() method applies the 6DoF
parallax effect, which is the difference in the apparent position
of an object when it is viewed from two different positions or viewpoints.
This method returns a promise.
The promise will be fulfilled with the processed image if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
Paster
The Paster interface provides methods to perform
sticker image pasting to a photo instance.
If failed to create, an InitFailureException will be thrown.
The getPlanesMap() method detects the detected planes in the
photo, in the form of an image mask. The mask pixels are index of
the planes (starting with 1.) A pixel value of zero indicates
areas without any detected plane surface.
This method returns a promise.
The promise will be fulfilled with the image instance with detected planes if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The setPhoto() method sets the photo that needs to be processed.
This method returns a promise.
The promise will be fulfilled if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The setSticker() specifies the sticker image, to be
pasted to the photo instance, and the pasting parameters.
Use the paste() function to perform the pasting operation.
This method returns a promise.
The promise will be fulfilled if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
StickerData for definitions.
PasteEffects for definitions.
The setSticker() method pastes the sticker image(s)
to the saved photo instance (by the SetPhoto function), and creates
the resulting photo. The saved photo is not modified.
Any subsequent pasting operations do not add up.
This method returns a promise.
The promise will be fulfilled with the resulting photo instance if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The setSticker() method creates an image mask to
preview how the sticker is to be pasted to the photo instance. The
image mask uses pixel value 255 for the foreground pixels and 0
for background pixels.
This method returns a promise.
The promise will be fulfilled with the image mask if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
Photo
The Photo interface manages a depth photo image
conforming to [[!XDM]] format.
The checkSignature() method checks the container image
Alder32 signature.
This method returns a promise.
If the container image is not modified, the promise will be fulfilled
with true. Otherwise, the promise will be fulfilled with false.
The promise will be rejected with the DOMException object if there is a failure.
The clone() method creates a new Photo instance and
copies the content of this photo to the new photo.
This method returns a promise.
The promise will be fulfilled with new photo if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The queryCameraPerspectiveModel() method gets the
specified camera perspective model parameters.
This method returns a promise.
The promise will be fulfilled with the specified camera perspective
model parameters if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The queryCameraPose() method gets the specified
camera pose information.
This method returns a promise.
The promise will be fulfilled with the specified camera pose
information if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The queryCameraVendorInfo() method gets the specified
camera vendor information.
This method returns a promise.
The promise will be fulfilled with the specified
camera vendor information if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The queryImage() method gets the color
image from the specified camera device (camera[cameraIndex]).
The unedited image is usually the unprocessed color image.
If cameraIndex is not specified,
the method gets the unedited color image in camera[0].
This method returns a promise.
The promise will be fulfilled with the color image if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The queryContainerImage() method gets the container
photo image.
This method returns a promise.
The promise will be fulfilled with the container photo image if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The queryDepth() method gets the depth
map of the photo from the specified camera (camera[cameraIndex]).
The depth map is the holefilled depth.
If cameraIndex is not specified, the method gets the depth image
in camera[0].
This method returns a promise.
The promise will be fulfilled with the depth image if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The queryDeviceVendorInfo() method gets the specified
device vendor information.
This method returns a promise.
The promise will be fulfilled with the specified
device vendor information if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The queryNumberOfCameras() method gets the number
of cameras.
This method returns a promise.
The promise will be fulfilled with the number of cameras if there
are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The queryRawDepth() method gets the raw depth image
of the photo. This would be the unprocessed depth captured from
the camera or loaded from a file if it existed.
This method returns a promise.
The promise will be fulfilled with the not-processed original
depth image if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The resetContainerImage() method copies the camera[0]
unedited color image to be the container image of the photo instance.
This method returns a promise.
The promise will be fulfilled if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The queryXDMRevision() method extracts the XDM file
revision string.
This method returns a promise.
The promise will be fulfilled with the XDM file revision string if
there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The setColorImage() method sets image to the color
image stored in the specified camera device. If cameraIndex is not
specified, the method sets the unedited color image in camera zero.
This method returns a promise.
The promise will be fulfilled if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The setContainerImage() method sets image to the
container photo image.
This method returns a promise.
The promise will be fulfilled if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The setDepthImage() method sets image to the depth
image stored in the specified camera device. If cameraIndex is not
specified, the method sets the depth image in camera zero.
This method returns a promise.
The promise will be fulfilled if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The setRawDepthImage() method sets image to the
not-processed original depth image
This method returns a promise.
The promise will be fulfilled if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
PhotoCapture
The PhotoCapture interface captures a depth photo
from 3D camera.
The MediaStream (described in [[!GETUSERMEDIA]]) passed
to the constructor must have at least one video track otherwise an
exception will be thrown.
If the MediaStream (described in [[!GETUSERMEDIA]]) passed
to the constructor is valid, but still failed to create the object,
an InitFailureException will be thrown.
The getDepthImage() method gets the latest
available depth image.
This method returns a promise.
The promise will be fulfilled with the latest available
depth image if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The takePhoto() method creates a photo instance
by importing content from preview color and depth images.
This method returns a promise.
The promise will be fulfilled with the photo instance if there are
no errors.
The promise will be rejected with the DOMException object if there is a failure.
The MediaStream instance passed in constructor.
A property used to set the EventHandler (described in [[!HTML]])
for the ErrorEvent that is dispatched
to PhotoCapture when an error happens
in preview.
A property used to set the EventHandler (described in [[!HTML]])
for the DepthQualityEvent that is dispatched
to PhotoCapture after checking the quality of
the depth map.
DepthQualityEvent interface
The depth map quality.
PhotoUtils
The PhotoUtils interface provides a set of utility
methods to manipulate a depth photo instance.
The colorResize() method resizes the color image of
the photo instance. The resized image maintains the same aspect
ratio as the original image. Thus only the target image width is
specified at the method input.
This method returns a promise.
The promise will be fulfilled with a color-resized photo instance if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The commonFOV() method creates a new photo instance
that matches the field of view of the color and depth images in the
input photo instance. Both the color and depth images are cropped
and camera meta data re-calculated.
This method returns a promise.
The promise will be fulfilled with the new photo instance if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The depthResize() method resizes the photo's depth map
to the specified size, which must have the same aspect ratio as
the original color image (thus only the width value is specified).
This method returns a promise.
The promise will be fulfilled with the depth map-resized photo image instance if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
DepthFillQuality enumerator for definition.
By default, the depth filling quality is high.
The enhanceDepth() method enhances the depth map
image quality by filling holes and performing denoising operations.
This method returns a promise.
The promise will be fulfilled with the depth enhanced photo image instance if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
DepthFillQuality enumerator for definition.
The getDepthQuality() method checks the depth map quality.
This method returns a promise.
The promise will be fulfilled with the depth map quality as defined
in the DepthMapQuality enumerator if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The photoCrop() crops the photo instance.
The depth map and camera meta data are updated accordingly.
This method returns a promise.
The promise will be fulfilled with the cropped photo image instance if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The photoRotate() performs 2D rotation of the photo
instance and creates the rotated photo instance. The rotation is
around the center of the color image clock wise.
The depth map and camera meta data are updated.
This method returns a promise.
The promise will be fulfilled with the rotated photo instance if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
Segmentation
The Segmentation interface provides methods to
perform depth-based object segmentation and refinement.
If failed to create, an InitFailureException will be thrown.
The objectSegment() generates an initial mask for
any object selected by the bounding mask. The mask can then be
refined by hints in the refineMask() method.
The mask image is in Y8 pixel format.
This method returns a promise.
The promise will be fulfilled with the mask image instance if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
Y8 pixel format.
The redo() method reapplies the last undo hints and generates
the mask image. The mask image is in Y8 pixel format.
This method returns a promise.
The promise will be fulfilled with the mask image instance if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The refineMask() method refines the image mask generated by
the objectSegment() method with the specified hint
points that indicate foreground or background pixels.
The mask image is in Y8 pixel format.
This method returns a promise.
The promise will be fulfilled with the refined mask image instance if there are no errors.
The returned image mask uses pixel value 255 for detected pixels and 0 for undetected pixels.
The promise will be rejected with the DOMException object if there is a failure.
The undo() method undoes the last hints and generates
the mask image.
The mask image is in Y8 pixel format.
This method returns a promise.
The promise will be fulfilled with the mask image instance if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
XDMUtils
The XDMUtils provides methods for [[!XDM]] content
in Blob instance (described in [[!FILEAPI]]).
The isXDM() method performs a quick check of the
specified blob data.
This method returns a promise.
The promise will be fulfilled with true if the blob data is in
[[!XDM]] format.
Otherwise, the promise will be fulfilled with false.
The promise will be rejected with the DOMException object if there is a failure.
The loadXDM() method reads the photo content from the
specified blob data in [[!XDM]] format.
This method returns a promise.
The promise will be fulfilled with the photo instance if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
The saveXDM() method writes the photo content to the
specified blob data in [[!XDM]] format.
This method returns a promise.
The promise will be fulfilled with the blob data if there are no errors.
The promise will be rejected with the DOMException object if there is a failure.
CameraPose
The translation vector.
The rotation quaternion.
CaptureConfiguration
The specified image width in pixels.
The specified image height in pixels.
The specified framerate.
MaskParams
The hint of the foreground object depth value.
The hint of the background object depth value.
The cut-off depth value for near end. Any object with the depth value smaller than the value is ignored in the processing.
The cut-off depth value for far end. Any object with the depth value bigger than the value is ignored in the processing.
MeasureData
The distance of two points in mm.
The confidence value from 0 (no confidence) to 1 (high confidence), NaN if not available.
The precision in mm, Nan if not available.
The starting point of the measurement.
The end point of the measurement.
Motion
Positive values indicate moving to the right.
Positive values indicate moving up.
Positive values indicate moving forward.
PasteEffects
If true, match the illumination of the background picture. By default this option is true.
Specify the transparency of the pasting operation. The value ranges from 0 (opaque, default) to 1.0f (complete transparent).
Specify the emboss effect high pass filter parameter. The value ranges from 0 (no emboss, default) to 1.0f (max emboss effect.)
If true, correct the pixel illumination taking account the background picture shading. By default, this option is false.
If true, add color correction. By default, this option is false.
PerspectiveCameraModel
The string that contains the camera name.
The focal length of the lens along the x/y-axis, normalized by the maximum dimension of the sensor.
The position indicating where the camera optical axis crosses the image plane center of the camera along the x/y-axis, normalized by the sensor resolution.
The skew of the image camera in degrees.
The radial distortion coefficients of the lens distortion.
The tangential distortion coefficients of the lens distortion.
Point
The x coordinate of the point.
The y coordinate of the point.
PreviewConfiguration
RadialDistortion
The radial distortion coefficients k1.
The radial distortion coefficients k2.
The radial distortion coefficients k3.
Rect
The horizontal coordinate of the top left pixel of the rectangle.
The vertical coordinate of the top left pixel of the rectangle.
The rectangle width in pixels.
The rectangle height in pixels.
Rotation
A positive pitch value indicates that rotates with X axis.
A positive yaw value indicates that rotates with Y axis.
A positive roll value indicates that clockwise rotates.
RotationQuaternion
Size
The rectangle width in pixels.
The rectangle height in pixels.
StickerData
The height of the sticker image in pixels.
Specify the image rotation, in degree. A positive value rotates the image clock-wise.
If true, the anchor of the sticker image is in the center of the image. If false, the anchor of the image is at the top left.
TangentialDistortion
The tangential distortion coefficients p1.
The tangential distortion coefficients p2.
TransitionVector
VendorInfo
The string that contains the camera or device name.
The manufacturer name.
Special notes about the device or camera device.
WorldPoint
The x coordinate of the point.
The y coordinate of the point.
The z coordinate of the point.
The confidence value from 0 (no confidence) to 1 (high confidence), NaN if not available.
The precision in mm, Nan if not available.
DepthFillQuality enum
Use the high quality depth filling algorithm for the case of offline photo processing.
Use the low quality depth filling algorithm for the case of real time live streaming.
DepthMapQuality enum
The depth map quality is poor.
The depth map quality is reasonably good for depth enabled photography functions but may lead to wrong results.
The depth map quality is good.
PixelFormat enum
The 32-bit RGBA32 color format. When format of
an Image instance is set to rgba32,
the data of that image instance must follow the
bytes layout of Canvas Pixel ArrayBuffer
defined in [[!CANVAS-2D]].
The depth map data in 16-bit unsigned integer. The values indicate the distance from an object to the camera's XY plane or the Cartesian depth.The value precision is in millimeters.
The depth map data in 32-bit floating point. The value precision is in millimeters.
The 8-bit gray format.
InitFailureException
var photoCapture;
navigator.mediaDevices.getUserMedia({video: true}).then(gotMedia, failedToGetMedia);
function gotMedia(stream) {
// Wire the media stream into a <video> element for preview.
var previewVideo = document.querySelector('#previewVideo');
previewVideo.srcObject = stream;
previewVideo.play();
// Construct a photo capture from the stream.
photoCapture = new realsense.DepthEnabledPhotography.PhotoCapture(stream);
photoCapture.onerror = function(e) {
console.log('Photo capture failure: ' + e.data.message);
};
photoCapture.ondepthquality = function(e) {
console.log('Depth quality: ' + e.data.quality);
};
}
var takePhotoButton = document.querySelector('#takePhoto');
takePhotoButton.onclick = function () {
photoCapture.takePhoto().then(gotPhoto, failedToGetPhoto);
};
function gotPhoto(photo) {
// Got a photo instance.
console.log('Got a photo');
}
// Access an <input> element with type='file' by using getElementById()
var loadPhoto = document.getElementById('loadPhoto');
var XDMUtils = realsense.DepthEnabledPhotography.XDMUtils;
loadPhoto.addEventListener('change', function(e) {
var file = loadPhoto.files[0];
XDMUtils.isXDM(file).then(
function(success) {
if (success) {
XDMUtils.loadXDM(file).then(
function(photo) {
// Got a photo instance
console.log('Got a photo');
},
function(e) { console.log(e.message); });
} else {
console.log ('This is not a XDM file. Load failed.');
}
},
function(e) { console.log(e.message); });
});
var url= 'http://path/to/depth/photo/test.jpg';
var xhr = new XMLHttpRequest();
xhr.open("GET", url, true);
xhr.responseType = 'blob';
xhr.onreadystatechange = function() {
if (xhr.readyState == 4 && xhr.status == 200) {
var blob = xhr.response;
XDMUtils.isXDM(blob).then(
function(success) {
if (success) {
XDMUtils.loadXDM(blob).then(
function(photo) {
// Got a photo instance
console.log('Got a photo');
},
function(e) { console.log(e.message); });
} else {
console.log('This is not a XDM file. Load failed.');
}
},
function(e) { console.log(e.message); });
}
};
xhr.send();
var refocus;
try {
refocus = new realsense.DepthEnabledPhotography.DepthRefocus();
} catch (e) {
console.log(e.message);
return;
}
// photo is the instance of DepthPhoto
// which can be got from the camera [Example 1] or loaded from somewhere [Example 2 or 3]
refocus.init(photo).then(
function() {
// [x, y] is the focusing point that you set.
// Here, the optional parameter aperture is omitted. The default is 50.0
refocus.apply({ x: x, y: y }).then(
function(refocusedPhoto) {
// Got the refocused photo
console.log('Got the refocused photo');
},
function(e) { console.log(e.message); });
},
function(e) { console.log(e.message); });
// photo is the instance of DepthPhoto
// which can be got from the camera [Example 1] or loaded from somewhere [Example 2 or 3]
photo.queryContainerImage().then(
function(colorImage) {
// The colorImage object follows the Image data structure, which could possibly
// looks like this:
// colorImage = { format: 'RGBA32',
// width: imageWidth,
// height: imageHeight,
// data: imageDataArrayBuffer
// };
// The imageDataArrayBuffer is an ArrayBuffer with type of Uint32Array, and the byte
// layout follows the Canvas Pixel ArrayBuffer defined in [CANVAS-2D].
},
function(error) { console.log(error.message); });
photo.checkSignature().then(
function(signature) {
console.log("signature: " + signature);
},
function(error) { console.log(error.message); });
photo.queryNumberOfCameras().then(
function(number) {
console.log("number of camera: " + number);
},
function(error) { console.log(error.message); });
photo.queryDeviceVendorInfo().then(
function(vendorInfo) {
console.log('model: ' + venterInfo.model + ', manufacturer: ' + venterInfo.manufacturer +
', notes: ' + venterInfo.notes);
},
function(error) { console.log(error.message); });
photo.queryCameraVendorInfo(0).then(
function(vendorInfo) {
console.log('model: ' + venterInfo.model + ', manufacturer: ' + venterInfo.manufacturer +
', notes: ' + venterInfo.notes);
},
function(error) { console.log(error.message); });
photo.queryCameraPerspectiveModel(0).then(
function(perspectiveModel) {
console.log('model: ' + perspectiveModel.model +
', focalLength: (' + perspectiveModel.focalLength.x + ', ' +
perspectiveModel.focalLength.y + ')' +
', principalPoint: (' + perspectiveModel.principalPoint.x + ', ' +
perspectiveModel.principalPoint.y + ')' +
', skew: ' + perspectiveModel.skew +
', radialDistortion: [' + perspectiveModel.radialDistortion.k1 + ', ' +
perspectiveModel.radialDistortion.k2 + ', ' +
perspectiveModel.radialDistortion.k3 + ']' +
', tangentialDistortion: [' + perspectiveModel.tangentialDistortion.p1 + ', ' +
perspectiveModel.tangentialDistortion.p2 + ']');
},
function(error) { console.log(error.message); });
photo.queryCameraPose(0).then(
function(pose) {
console.log('transition: (' + pose.transition.x + ', ' +
pose.transition.y + ', ' +
pose.transition.z + ')' +
', rotation: (' + pose.rotation.rotationAngle + ', ' +
pose.rotation.rotationAxisX + ', ' +
pose.rotation.rotationAxisY + ', ' +
pose.rotation.rotationAxisZ + ')');
},
function(error) { console.log(error.message); });
photo.queryImage().then(
function(colorImage) {
// Render the color image.
},
function(error) { console.log(error.message); });
photo.queryDepth().then(
function(depthImage) {
// The depthImage object follows the Image data structure, which could possibly
// looks like this:
// depthImage = { format: 'DEPTH' / 'DEPTH_F32',
// width: imageWidth,
// height: imageHeight,
// data: imageDataArrayBuffer
// };
// The imageDataArrayBuffer is an ArrayBuffer with type of Uint16Array(for DEPTH)
// or Float32Array(for DEPTH_F32), and the value indicates the distance from an object
// to the camera's XY plane or the Cartesian depth. The value precision is millimeters.
},
function(error) { console.log(error.message); });
photo.queryRawDepth().then(
function(rawDepthImage) {
// Render the raw depth image.
},
function(error) { console.log(error.message); });
photo.resetContainerImage().then(
function() {
// Cleanup work after reset.
},
function(error) { console.log(error.message); });
// photo is the instance of DepthPhoto
// which can be got from the camera [Example 1] or loaded from somewhere [Example 2 or 3]
var photoUtils = realsense.DepthEnabledPhotography.PhotoUtils;
function colorResize() {
// Resize photo's color image to 1280px width.
photoUtils.colorResize(photo, 1280).then(
function(resizedPhoto) {
resizedPhoto.queryContainerImage().then(
function(image) {
// Render your resized image.
},
function(error) { console.log(error.message); });
},
function(error) { console.log(error.message); });
}
function commonEOV() {
photoUtils.commonFOV(photo).then(
function(processedPhoto) {
processedPhoto.queryContainerImage().then(
function(image) {
// Render your commonFOV processed photo.
},
function(error) { console.log(error.message); });
},
function(error) { console.log(error.message); });
}
function depthEnhance() {
var before = new Date().getTime();
photoUtils.enhanceDepth(photo, 'high').then(
function(enhancedPhoto) {
var after = new Date().getTime();
var diff = after - before;
enhancedPhoto.queryDepth().then(
function(image) {
// Render the depth enhanced photo image.
},
function(error) { console.log(error.message); });
},
function(error) { console.log(error.message); });
}
function depthResize() {
// Resize photo's depth image to 640px width.
photoUtils.depthResize(photo, 640).then(
function(resizedPhoto) {
resizedPhoto.queryDepth().then(
function(image) {
// Render the resized depth photo image.
},
function(error) { console.log(error.message); });
},
function(error) { console.log(error.message); });
}
function depthQuality() {
photoUtils.getDepthQuality(photo).then(
function(quality) {
console.log('Depth qulity = ' + quality);
},
function(error) { console.log(error.message); });
}
function photoCrop() {
// Use originX, originY, width, height to define the bounding rectangle of the crop area.
photoUtils.photoCrop(photo, { x: originX, y: originY, w: width, h: height}).then(
function(croppedPhoto) {
croppedPhoto.queryContainerImage().then(
function(image) {
// Render the cropped photo image.
},
function(error) { console.log(error.message); });
},
function(error) { console.log(error.message); });
}
function photoRotate() {
// Rotate photo by 90 degree clockwise.
photoUtils.photoRotate(photo, 90.0).then(
function(rotatedPhoto) {
rotatedPhoto.queryContainerImage().then(
function(image) {
// Render the rotated photo image.
},
function(error) { console.log(error.message); });
},
function(error) { console.log(error.message); });
}
// photo is the instance of DepthPhoto
// which can be got from the camera [Example 1] or loaded from somewhere [Example 2 or 3]
var measurement;
try {
measurement = new realsense.DepthEnabledPhotography.Measurement();
} catch (error) {
console.log(error.message);
return;
}
// startX, startY, endX and endY defines the start and end point of the measurement respectively.
measurement.measureDistance(photo, { x: startX, y: startY }, { x: endX, y: endY }).then(
function(d) {
console.log(
'Distance between(' + startX + ',' + startY +
') - (' + endX + ',' + endY + ') = ' +
parseFloat(d.distance).toFixed(2) + ' millimeters, Confidence = ' +
parseFloat(d.confidence).toFixed(2) + ', Precision=' +
parseFloat(d.precision).toFixed(2) + 'mm');
},
function(error) { console.log(error.message); });
var motionEffect;
try {
motionEffect = new realsense.DepthEnabledPhotography.MotionEffect();
} catch (error) {
console.log(error.message);
return;
}
motionEffect.init(photo).then(
function() {
doMothionEffect();
},
function(error) { console.log(error.message); });
function doMothionEffect() {
// Set these motion parameters at your convenience. In this demo, we moved
// the photo 40px to the right and zoomed it out 1.5 times.
var right = 40.0, up = 0.0, forward = 0.0;
var yaw = 0.0, pitch = 0.0, roll = 0.0;
var zoom = 1.5;
motionEffect.apply({ horizontal: right, vertical: up, distance: forward },
{ pitch: pitch, yaw: yaw, roll: roll },
zoom).then(
function(image) {
// Render the motion effect applied photo image.
},
function(error) { console.log(error.message); });
}
var segmentation;
try {
segmentation = new realsense.DepthEnabledPhotography.Segmentation();
} catch (error) {
console.log(error.message);
return;
}
var markupImageBuffer = []; // An ArrayBuffer for bounding mask image data
// (photo.width x photo.height), pixel value should be
// 255 for foreground and 0 for background.
// Cleanup the markupImageBuffer
var len = photo.with * photo.height;
for (var i = 0; i < len; i++) {
markupImageBuffer[i] = 0;
}
(function createInitialMask(topX, topY, bottomX, bottomY) {
var value = 255;
for (var y = topY; y != bottomY; y += (topY < bottomY) ? 1 : -1) {
for (var x = topX; x != bottomX; x += (topX < bottomX) ? 1 : -1) {
// Set the pixel value to 255 within the bounding area of (topX, topY, buttomX, buttomY).
markupImageBuffer[y * photo.width + x] = value;
}
}
})(10, 10, 100, 100); // Created a mask image with bounding rect of (10, 10, 100, 100).
var initialMarkupImage = {
format: 'Y8',
width: photo.width,
height: photo.height,
data: markupImageBuffer
};
segmentation.objectSegment(photo, initialMarkupImage).then(
function(maskImage) {
// As the segmentation.refineMask() needs to be called after a successful call of
// segmentation.objectSegment(), here we dispatch an event 'objectSegmentSucceed'
// to notify a successful calling.
window.dispatchEvent(new Event('objectSegmentSucceed'));
photo.queryContainerImage().then(
function(colorImage) {
// Blend and render the colorImage and maskImage.
},
function(error) { console.log(error.message); });
},
function(error) { console.log(error.message); });
window.addEventListener('objectSegmentSucceed', function(event) {
// The maskArea is always generated by user action (e.g. a serie of mouse click), you
// could use a event listener to capture the mouse click on your canvas.
var maskArea = [{x: 0, y: 0}, {x: 10, y: 10}, ...];
var isForeground = true; // Indicate if the refinement take place in forground or background.
segmentation.refineMask(maskArea, isForeground).then(
function(maskImage) {
photo.queryContainerImage().then(
function(colorImage) {
// Blend and render the colorImage and maskImage;
},
function(error) { console.log(error.message); });
},
function(error) { console.log(error.message); });
}, false);