Project Trouble Shooting : NonUniqueResultException by JPA - Using Imagehash to prevent uploading duplicated Images

#Foreword

While implementing File Image APIs with JPA and MySQL, I've encountered various issues. Specifically, when uploading file images with the POST method, i faced file size limit problems (this document will not cover this, but I've added reference URLs).

Initially, I thought resolving the file size limit issues would be sufficient. However, even after resolving those problems, I encountered another issue when uploading and retrieving images.

When duplicate images are saved in the database, trying to retrieve images results in a 'NonUniqueResultException' with the message: "Query did not return a unique result." Let's dive into how I solved this problems.

Reference URL : File Image APIs and API Design

: https://joo.hashnode.dev/project-file-image-apis-with-jpamysql-api-design

Reference URL : MysqlDataTruncation, FileSizeExceededException

: https://joo.hashnode.dev/project-trouble-shooting-file-image-upload-file-size-problems-with-mysql-tomcat

#Problem : NonUniqueResultException by JPA

: When uploading images files, Program fails to block uploading of duplicate Image files. This causes issues with retrieving specific images from the database.

#Duplicate Images cause stacking issues (Snapshot)

Currently, when using POST Mapping, duplicate images are stacked in the database without updating existing files on every request


After uploading images, I couldn't retrieve specific Image with the error below.

If there's only one image -> runs Well!

Error again.

#What Things Can Serve as a Unique Identifier for an Image File (byte[])

When a POST request is made to upload an image file, the same file name can lead to stacking issues, meaning the file name cannot serve as a unique identifier.

Instead, image files are saved as byte arrays ('byte[]'). This approach ensures that the uniqueness of an image is not tied to its file name or other superficial attributes.

To uniquely identify each image, we can generate a hash of the image content. This hash acts as a unique identifier, allowing us to manage and retrieve images accurately without relying on the file name or anything else.

#Solution : add Image Hash

Reference URL : File Image APIs and API Design

: https://joo.hashnode.dev/project-file-image-apis-with-jpamysql-api-design

@Service
@RequiredArgsConstructor
public class ImageDataService {

    private final ImageDataRepository imageDataRepository;
    private final MemberRepository memberRepository;
    private final FreeBoardRepository freeBoardRepository;
    private final OfficeRepository officeRepository;
    private final PlaceRepository placeRepository;

    public ImageDataResponse uploadImage(MultipartFile multipartFile, String entityType, Long entityId, HttpServletRequest httpServletRequest)
        throws IOException, NoSuchAlgorithmException {

        // Generate a hash for the image to prevent duplicate uploads
        String imageHash = generateImageHash(multipartFile.getBytes());
        Optional<ImageData> existingImage = imageDataRepository.findByHash(imageHash);
        if (existingImage.isPresent()) {
            throw new ImageFileAlreadyRegisteredException();
        }

        // Construct the base URL for the image
        String baseUrl = String.format(
            "%s://%s:%d/images",
            httpServletRequest.getScheme(),
            httpServletRequest.getServerName(),
            httpServletRequest.getServerPort()
        );

        // Initialize entities
        Member member = null;
        FreeBoard freeBoard = null;
        Office office = null;
        Place place = null;

        // Determine the entity type and retrieve the corresponding entity
        if ("member".equalsIgnoreCase(entityType)) {
            member = memberRepository.findById(entityId)
                .orElseThrow(() -> new MemberNotFoundException());
        } else if ("freeBoard".equalsIgnoreCase(entityType)) {
            freeBoard = freeBoardRepository.findById(entityId)
                .orElseThrow(() -> new FreeBoardNotFoundException());
        } else if ("office".equalsIgnoreCase(entityType)) {
            office = officeRepository.findById(entityId)
                .orElseThrow(() -> new OfficeNotFoundException());
        } else if ("place".equalsIgnoreCase(entityType)) {
            place = placeRepository.findById(entityId)
                .orElseThrow(() -> new PlaceNotFoundException());
        }

        // Save new image data to the database if it does not already exist
        ImageData savedNewImageData = imageDataRepository.save(
            ImageData.builder()
                .name(multipartFile.getOriginalFilename())
                .type(multipartFile.getContentType())
                .hash(imageHash)
                .imageData(ImageUtils.compressImage(multipartFile.getBytes()))
                .url(baseUrl + "/" + multipartFile.getOriginalFilename())  // Construct the URL
                .member(member)
                .freeBoard(freeBoard)
                .office(office)
                .place(place)
                .build()
        );

        // Add the image to the respective entity's image collection
        if (member != null) {
            member.getImages().add(savedNewImageData);
        }
        if (freeBoard != null) {
            freeBoard.getImages().add(savedNewImageData);
        }
        if (office != null) {
            office.getImages().add(savedNewImageData);
        }
        if (place != null) {
            place.getImages().add(savedNewImageData);
        }

        return ImageDataResponse.from(savedNewImageData);
    }

    // Generate a SHA-256 hash for the given image data to prevent duplicate uploads
    private String generateImageHash(byte[] imageData) throws NoSuchAlgorithmException {
        MessageDigest digest = MessageDigest.getInstance("SHA-256");
        byte[] hashBytes = digest.digest(imageData);
        StringBuilder sb = new StringBuilder();
        for (byte b : hashBytes) {
            sb.append(String.format("%02x", b));
        }
        return sb.toString();
    }
}

#Solution Test Result - POST

#Solution Test Result - DELETE

DELETE - One more request.


For my future reference (Issue)

Even though I resolved NonUniqueResultException by adding image hash, but different Requests can also blocked with this image hash. Frankly speaking, I missed this during the project, The reason I wrote this down on this, I would've refered to this description.