java实现简单的oss存储

提示：扫一扫查出行
【扫一扫了解最新限行尾号】
复制提示

oss

工作中需要用到文件上传，之前使用的是本地文件系统存储方式，后来重构为支持多个存储源的方式，目前支持三种方式：local、seaweedfs、minio

存储介质

seaweedfs

seaweedfs是一款go语言开发的轻量级、高性能的存储服务器。

https://github.com/chrislusf/seaweedfs

# 启动 master

docker run -d \

-p 9333:9333 \

-p 19333:19333 \

-v /home/mining/report-cloud/docker/seaweedfs/master/data:/data \

--name seaweedfs-master \

chrislusf/seaweedfs:1.53 \

master -ip=master

# 启动 volume

docker run -d \

-p 8080:8080 \

-p 18080:18080 \

-v /home/mining/report-cloud/docker/seaweedfs/volume01/data:/data \

--name seaweedfs-volume \

--link seaweedfs-master:master \

chrislusf/seaweedfs:1.53 \

volume -max=5 -mserver="master:9333" -port=8080

访问web地址：http://localhost:9333/

简单使用：

# 申请空间

curl http://localhost:9333/dir/assign

#

{"count":1,"fid":"4,017f52110b","url":"127.0.0.1:8080","publicUrl":"localhost:80

80"}

# 推送文件，4,017f52110b表示第4个volume，017f52110b表示文件的唯一标识

curl -F file=@/home/seaweedfs/balance.png http://127.0.0.1:8080/4,017f52110b

minio

MinIO是与Amazon S3 API兼容的高性能对象存储服务器，提供了人性化的管理页面

https://github.com/minio/minio

docker run --name report-minio \

-p 19000:9000 \

-e "MINIO_ACCESS_KEY=qweasdzxc" \

-e "MINIO_SECRET_KEY=1234567890" \

-v /home/mining/report-cloud/docker/minio:/data \

-d minio/minio:latest server /data

# web 管理页面

http://localhost:19000/minio/login

# 创建一个 bucket 为 report

# 通过URL直接访问文件需要设置权限，参考下面博客

# https://blog.csdn.net/iKaChu/article/details/105809957

# 访问格式为

http://localhost:19000/bucket名/对象名

Java代码

maven依赖

<!-- oss 存储的依赖 -->

<!-- seaweedfs 依赖 -->

<dependency>

  <groupId>org.lokra.seaweedfs</groupId>

  <artifactId>seaweedfs-client</artifactId>

  <version>0.7.3.RELEASE</version>

</dependency>

<!-- minio 依赖 -->

<dependency>

  <groupId>io.minio</groupId>

  <artifactId>minio</artifactId>

  <version>3.0.10</version>

</dependency>

代码

配置类

FileTag 枚举类

用来标记使用哪种存储介质

/**

 * 文件上传的tag

 */

public enum FileTag {

    TYPE_LOCAL,

    TYPE_MINIO ,

    TYPE_SEAWEEDFS;

}

FileUploadConfigLocal

本地存储配置类

public class FileUploadConfigLocal {

    private String parentPath;

    // 省略 get set

}

FileUploadConfigSeaweedfs

seaweedfs存储配置类

public class FileUploadConfigSeaweedfs {

    private String host;

    private int port = 9333;

    private String publicUrl;

    private int connectTimeout = 60; // 60 s

    // 省略 get set

}

FileUploadConfigMinio

minio存储配置类

public class FileUploadConfigMinio {

    private String domain; // url 前缀

    private String accesKey;

    private String secretKey;

    private String bucket; // 必须事先创建好

    // 省略 get set

}

FileUploadConfig

spring application.yaml 配置类

import org.springframework.boot.context.properties.ConfigurationProperties;

@ConfigurationProperties( prefix = "oss-config")

public class FileUploadConfig {

    FileTag tag;

    FileUploadConfigLocal local;

    FileUploadConfigSeaweedfs seaweedfs;

    FileUploadConfigMinio minio;

    // 省略 get set

}

配置文件

application.yaml

# 对象存储配置

oss-config:

  tag: TYPE_LOCAL # 使用哪种存储介质

  local:

    parent-path: D:\dev\2019-06-19\upload

  minio:

    domain: http://localhost:19000

    bucket: report

    acces-key: qweasdzxc

    secret-key: 1234567890

  seaweedfs:

    host: localhost

    port: 9333

    publicUrl: http://localhost:8080

    connectTimeout: 3000 # 5min

FileUploadInfoVO文件上传类

public class FileUploadInfoVO {

    private String fileName; // 文件名

    private String suffix; // 文件后缀

    private String contentType; // 文件类型

    private Long size; // 文件大小

    private String fid; // 文件唯一ID，同时也是保存在服务器上的文件名

    private boolean delete; // 是否删除，默认false

    private String description;

    private String url; // 完整的url

    private String tag; // 标签，比如

    private String uploadTime;

    private String deleteTime;

    // 省略 get set

}

业务逻辑类

FileStorageService 文件存储接口

/**

 * 文件存储的接口

 */

public interface FileStorageService {

    FileUploadInfoVO upload(InputStream in, FileUploadInfoVO fileUploadInfoVO) throws Exception ;

    FileUploadInfoVO upload(byte[] bytes, FileUploadInfoVO fileUploadInfoVO) throws Exception ;

    FileUploadInfoVO delete(FileUploadInfoVO fileUploadInfoVO) throws Exception ;

}

本地存储实现

@Service("FileStorageServiceLocal")

public class FileStorageServiceLocal implements FileStorageService {

    @Autowired

    FileUploadConfig fileUploadConfig;

    @PostConstruct

    public void init() {

        FileUtils.checkAndCreateDir(fileUploadConfig.getLocal().getParentPath());

    }

    @Override

    public FileUploadInfoVO upload(InputStream in, FileUploadInfoVO fileUploadInfoVO) throws Exception {

        String filePath = fileUploadConfig.getLocal().getParentPath() + File.separator + fileUploadInfoVO.getFid();

        FileUtils.copyFileFromInputStream(in, filePath);

        fileUploadInfoVO.setUrl(filePath+"."+fileUploadInfoVO.getSuffix());

        return fileUploadInfoVO;

    }

    @Override

    public FileUploadInfoVO upload(byte[] bytes, FileUploadInfoVO fileUploadInfoVO) throws Exception {

        String filePath = fileUploadConfig.getLocal().getParentPath() + File.separator + fileUploadInfoVO.getFid();

        FileUtils.copyFileFromBytes(bytes, filePath);

        fileUploadInfoVO.setUrl(filePath);

        return fileUploadInfoVO;

    }

    @Override

    public FileUploadInfoVO delete(FileUploadInfoVO fileUploadInfoVO) throws Exception {

        File file = new File(fileUploadInfoVO.getUrl());

        if(file.exists()) {

            file.delete();

        }

        return fileUploadInfoVO;

    }

}

Seaweedfs存储实现

@Service("FileStorageServiceSeaweedfs")

public class FileStorageServiceSeaweedfs implements FileStorageService {

    @Autowired

    FileUploadConfig fileUploadConfig;

    FileTemplate fileTemplate;

    @PostConstruct

    public void init() {

        FileUploadConfigSeaweedfs seaweedfs = fileUploadConfig.getSeaweedfs();

        FileSource fileSource = new FileSource();

        fileSource.setHost(seaweedfs.getHost());

        fileSource.setPort(seaweedfs.getPort());

        fileSource.setConnectionTimeout(seaweedfs.getConnectTimeout());//5min

        try {

            fileSource.startup();

        } catch (IOException e) {

            throw new RuntimeException("创建seaweedfs连接失败，原因是：" + e.getMessage());

        }

        fileTemplate = new FileTemplate(fileSource.getConnection(), seaweedfs.getPublicUrl());

    }

    @Override

    public FileUploadInfoVO upload(InputStream in, FileUploadInfoVO fileUploadInfoVO) throws Exception {

        FileHandleStatus fileHandleStatus = fileTemplate.saveFileByStream(fileUploadInfoVO.getFileName(), in, ContentType.create(fileUploadInfoVO.getContentType(), "utf-8"));

        fileUploadInfoVO.setFid(fileHandleStatus.getFileId());

        fileUploadInfoVO.setUrl(String.format("%s/%s", fileUploadConfig.getSeaweedfs().getPublicUrl(), fileHandleStatus.getFileId()));

        return fileUploadInfoVO;

    }

    @Override

    public FileUploadInfoVO upload(byte[] bytes, FileUploadInfoVO fileUploadInfoVO) throws Exception {

        return upload(new ByteArrayInputStream(bytes), fileUploadInfoVO);

    }

    @Override

    public FileUploadInfoVO delete(FileUploadInfoVO fileUploadInfoVO) throws Exception {

        fileTemplate.deleteFile(fileUploadInfoVO.getFid());

        return fileUploadInfoVO;

    }

}

Minio存储实现

@Service("FileStorageServiceMinio")

public class FileStorageServiceMinio implements FileStorageService {

    @Autowired

    FileUploadConfig fileUploadConfig;

    @Override

    public FileUploadInfoVO upload(InputStream in, FileUploadInfoVO fileUploadInfoVO) throws Exception {

        FileUploadConfigMinio minio = fileUploadConfig.getMinio();

        String domain = minio.getDomain();

        String bucket = minio.getBucket();

        String fileName = fileUploadInfoVO.getFileName();

        MinioClient minioClient = new MinioClient(domain, minio.getAccesKey(), minio.getSecretKey());

        minioClient.putObject("report", fileName, in, fileUploadInfoVO.getSize(), fileUploadInfoVO.getContentType());

        fileUploadInfoVO.setUrl(String.format("%s/%s/%s", domain, bucket, fileName));

        return fileUploadInfoVO;

    }

    @Override

    public FileUploadInfoVO upload(byte[] bytes, FileUploadInfoVO fileUploadInfoVO) throws Exception {

        return upload(new ByteArrayInputStream(bytes), fileUploadInfoVO);

    }

    @Override

    public FileUploadInfoVO delete(FileUploadInfoVO fileUploadInfoVO) throws Exception {

        FileUploadConfigMinio minio = fileUploadConfig.getMinio();

        MinioClient minioClient = new MinioClient(minio.getDomain(), minio.getAccesKey(), minio.getSecretKey());

        minioClient.removeObject(minio.getBucket(), fileUploadInfoVO.getFileName());

        return fileUploadInfoVO;

    }

}

存储工厂类实现

用来根据 FileTag 枚举类获得对应的具体实现

@Service

public class FileStorageFactory {

    Map<FileTag, FileStorageService> fileStorageServiceMap;

    @Qualifier("FileStorageServiceLocal")

    @Autowired

    FileStorageService fileStorageServiceLocal;

    @Qualifier("FileStorageServiceSeaweedfs")

    @Autowired

    FileStorageService fileStorageServiceSeaweedfs;

    @Qualifier("FileStorageServiceMinio")

    @Autowired

    FileStorageService fileStorageServiceMinio;

    @PostConstruct

    public void init() {

        fileStorageServiceMap = new HashMap<>();

        fileStorageServiceMap.put(FileTag.TYPE_LOCAL, fileStorageServiceLocal);

        fileStorageServiceMap.put(FileTag.TYPE_SEAWEEDFS, fileStorageServiceSeaweedfs);

        fileStorageServiceMap.put(FileTag.TYPE_MINIO, fileStorageServiceMinio);

    }

    public FileStorageService get(FileTag type) {

        FileStorageService fileStorageService = fileStorageServiceMap.get(type);

        return fileStorageService == null ? fileStorageServiceMap.get(FileTag.TYPE_LOCAL) : fileStorageService;

    }

}

业务中实现

这里根据具体的业务，来选择具体的实现逻辑

eg：

@Autowired

private FileStorageFactory fileStorageFactory;

public FileUploadInfoVO put(MultipartFile multipartFile) throws Exception {

  FileUploadInfoVO vo = new FileUploadInfoVO();

  // 填充一些信息（省略）

  // 获得文件上传处理器

  FileStorageService fileStorageService = fileStorageFactory.get(FileTag.TYPE_LOCAL);

  fileUploadInfoVO = fileStorageService.upload(multipartFile.getInputStream(), vo);

  // ... 业务逻辑

}

这样，配置文件中可以使用FileTag来控制文件存储的介质，从而实现一个简单的对象存储服务器。

正文

oss

存储介质

seaweedfs

minio

Java代码

maven依赖

代码

配置类

FileTag 枚举类

FileUploadConfigLocal

FileUploadConfigSeaweedfs

FileUploadConfigMinio

FileUploadConfig

配置文件

FileUploadInfoVO文件上传类

业务逻辑类

FileStorageService 文件存储接口

本地存储实现

Seaweedfs存储实现

Minio存储实现

存储工厂类实现

业务中实现

相关阅读

阿里云服务器配合oss，阿里云服务器挂载oss

oss阿里服务器，阿里云os云服务

阿里云Oss还是服务器，阿里云 oss sts

虚拟主机配置hosts文件，Hosts文件配置存在问题

游戏开发优先，游戏开发前应考虑哪些因素?

飞狐棋牌游戏的简单介绍

广电app如何电商，广电怎么赚钱

erp系统会用到函数吗，erp系统用到哪些技术

目录[+]