Java GenericObjectPool 对象池化技术之SpringBoot sftp 连接池工具类详解
作者:VipSoft
Java BasePooledObjectFactory 对象池化技术
通常一个对象创建、销毁非常耗时的时候,我们不会频繁的创建和销毁它,而是考虑复用。复用对象的一种做法就是对象池,将创建好的对象放入池中维护起来,下次再用的时候直接拿池中已经创建好的对象继续用,这就是池化的思想。
Apache Commons Pool是一个对象池的框架,他提供了一整套用于实现对象池化的API。它提供了三种对象池:GenericKeyedObjectPool,SoftReferenceObjectPool和GenericObjectPool,其中GenericObjectPool是我们最常用的对象池,内部实现也最复杂。
GenericObjectPool
GenericObjectPool 是一个通用对象池框架,我们可以借助它实现一个健壮的对象池,UML图如下所示:
GenericObjectPool 实现了ObjectPool接口,而ObjectPool就是对象池的核心接口,它定义了一个对象池应该实现的行为。
public interface ObjectPool<T> extends Closeable { /** * 从池中借走到一个对象 */ T borrowObject() throws Exception, NoSuchElementException, IllegalStateException; /** * 把对象归还给对象池 */ void returnObject(T var1) throws Exception; /** * 验证对象的有效性 */ void invalidateObject(T var1) throws Exception; /** * 往池中添加一个对象 */ void addObject() throws Exception, IllegalStateException, UnsupportedOperationException; /** * 返回对象池中有多少对象是空闲的,也就是能够被借走的对象的数量。 */ int getNumIdle(); /** * 返回对象池中有对象对象是活跃的,也就是已经被借走的,在使用中的对象的数量。 */ int getNumActive(); /** * 清理对象池。注意是清理不是清空,该方法要求的是,清理所有空闲对象,释放相关资源。 */ void clear() throws Exception, UnsupportedOperationException; /** * 关闭对象池。这个方法可以达到清空的效果,清理所有对象以及相关资源。 */ void close(); }
BasePooledObjectFactory
Java BasePooledObjectFactory 对象池化技术
使用GenericObjectPool
只需要创建一个对象工厂类,继承BasePooledObjectFactory
并重写它的create()
和destroyObject()
。
如下文中的:SftpPool.java
public interface PooledObjectFactory<T> { /** * 创建一个可由池提供服务的实例,并将其封装在由池管理的PooledObject中。 */ PooledObject<T> makeObject() throws Exception; /** * 销毁池不再需要的实例 */ void destroyObject(PooledObject<T> var1) throws Exception; /** * 确保实例可以安全地由池返回 */ boolean validateObject(PooledObject<T> var1); /** * 重新初始化池返回的实例 */ void activateObject(PooledObject<T> var1) throws Exception; /** * 取消初始化要返回到空闲对象池的实例 */ void passivateObject(PooledObject<T> var1) throws Exception; }
配置类GenericObjectPoolConfig
GenericObjectPoolConfig
是封装GenericObject
池配置的简单“结构”,此类不是线程安全的;它仅用于提供创建池时使用的属性。大多数情况,可以使用GenericObjectPoolConfig
提供的默认参数就可以满足日常的需求。
工作原理流程
- 构造方法
当我们执行构造方法时,主要工作就是创建了一个存储对象的LinkedList类型容器,也就是概念意义上的“池” - 从对象池中获取对象
获取池中的对象是通过borrowObject()命令,源码比较复杂,简单而言就是去LinkedList中获取一个对象,如果不存在的话,要调用构造方法中第一个参数Factory工厂类的makeObject()方法去创建一个对象再获取,获取到对象后要调用validateObject方法判断该对象是否是可用的,如果是可用的才拿去使用。LinkedList容器减一 - 归还对象到线程池
简单而言就是先调用validateObject方法判断该对象是否是可用的,如果可用则归还到池中,LinkedList容器加一,如果是不可以的则调用destroyObject方法进行销毁
上面三步就是最简单的流程,由于取和还的流程步骤都在borrowObject和returnObject方法中固定的,所以我们只要重写Factory工厂类的makeObject()和validateObject以及destroyObject方法即可实现最简单的池的管理控制,通过构造方法传入该Factory工厂类对象则可以创建最简单的对象池管理类。这算是比较好的解耦设计模式,借和还的流程如下图所示:
使用Demo
<dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-pool2</artifactId> <version>2.7.0</version> </dependency> <!-- https://mvnrepository.com/artifact/com.jcraft/jsch --> <dependency> <groupId>com.jcraft</groupId> <artifactId>jsch</artifactId> <version>0.1.55</version> </dependency>
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>vipsoft-parent</artifactId> <groupId>com.vipsoft.boot</groupId> <version>1.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>vipsoft-sftp</artifactId> <version>1.0-SNAPSHOT</version> <dependencies> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-pool2</artifactId> <version>2.7.0</version> </dependency> <!-- https://mvnrepository.com/artifact/com.jcraft/jsch --> <dependency> <groupId>com.jcraft</groupId> <artifactId>jsch</artifactId> <version>0.1.55</version> </dependency> <dependency> <groupId>org.eclipse.paho</groupId> <artifactId>org.eclipse.paho.client.mqttv3</artifactId> <version>1.2.5</version> </dependency> <dependency> <groupId>cn.hutool</groupId> <artifactId>hutool-all</artifactId> <version>5.3.6</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> <exclusions> <exclusion> <groupId>org.junit.vintage</groupId> <artifactId>junit-vintage-engine</artifactId> </exclusion> </exclusions> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build> </project>
application.yaml
server: port: 8088 application: name: sftp Demo sftp: host: 172.16.3.88 # 服务器ip port: 22 # ssh端口 username: root # 用户名 password: root # 密码 # 连接池参数 pool: max-total: 10 max-idle: 10 min-idle: 5
SftpPoolException.java
package com.vipsoft.sftp.exception; /** * sftp连接池异常 */ public class SftpPoolException extends RuntimeException { private static final long serialVersionUID = 1L; /** * Constructs a new runtime exception with {@code null} as its * detail message. The cause is not initialized, and may subsequently be * initialized by a call to {@link #initCause}. */ public SftpPoolException() { } /** * Constructs a new runtime exception with the specified detail message. * The cause is not initialized, and may subsequently be initialized by a * call to {@link #initCause}. * * @param message the detail message. The detail message is saved for * later retrieval by the {@link #getMessage()} method. */ public SftpPoolException(String message) { super(message); } /** * Constructs a new runtime exception with the specified detail message and * cause. <p>Note that the detail message associated with * {@code cause} is <i>not</i> automatically incorporated in * this runtime exception's detail message. * * @param message the detail message (which is saved for later retrieval * by the {@link #getMessage()} method). * @param cause the cause (which is saved for later retrieval by the * {@link #getCause()} method). (A <tt>null</tt> value is * permitted, and indicates that the cause is nonexistent or * unknown.) * @since 1.4 */ public SftpPoolException(String message, Throwable cause) { super(message, cause); } /** * Constructs a new runtime exception with the specified cause and a * detail message of <tt>(cause==null ? null : cause.toString())</tt> * (which typically contains the class and detail message of * <tt>cause</tt>). This constructor is useful for runtime exceptions * that are little more than wrappers for other throwables. * * @param cause the cause (which is saved for later retrieval by the * {@link #getCause()} method). (A <tt>null</tt> value is * permitted, and indicates that the cause is nonexistent or * unknown.) * @since 1.4 */ public SftpPoolException(Throwable cause) { super(cause); } /** * Constructs a new runtime exception with the specified detail * message, cause, suppression enabled or disabled, and writable * stack trace enabled or disabled. * * @param message the detail message. * @param cause the cause. (A {@code null} value is permitted, * and indicates that the cause is nonexistent or unknown.) * @param enableSuppression whether or not suppression is enabled * or disabled * @param writableStackTrace whether or not the stack trace should * be writable * @since 1.7 */ public SftpPoolException(String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace) { super(message, cause, enableSuppression, writableStackTrace); } }
config
SftpConfig.java
package com.vipsoft.sftp.config; import com.vipsoft.sftp.pool.SftpFactory; import com.vipsoft.sftp.pool.SftpPool; import com.vipsoft.sftp.utils.SftpUtil; import org.springframework.boot.context.properties.EnableConfigurationProperties; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @Configuration @EnableConfigurationProperties(SftpProperties.class) public class SftpConfig { // 工厂 @Bean public SftpFactory sftpFactory(SftpProperties properties) { return new SftpFactory(properties); } // 连接池 @Bean public SftpPool sftpPool(SftpFactory sftpFactory) { return new SftpPool(sftpFactory); } // 辅助类 @Bean public SftpUtil sftpUtil(SftpPool sftpPool) { return new SftpUtil(sftpPool); } }
SftpProperties.java
package com.vipsoft.sftp.config; import com.jcraft.jsch.ChannelSftp; import org.apache.commons.pool2.impl.GenericObjectPoolConfig; import org.springframework.boot.context.properties.ConfigurationProperties; @ConfigurationProperties(prefix = "sftp") public class SftpProperties { private String host; private int port = 22; private String username = "root"; private String password = "root"; private Pool pool = new Pool(); public String getHost() { return host; } public void setHost(String host) { this.host = host; } public int getPort() { return port; } public void setPort(int port) { this.port = port; } public String getUsername() { return username; } public void setUsername(String username) { this.username = username; } public String getPassword() { return password; } public void setPassword(String password) { this.password = password; } public Pool getPool() { return pool; } public void setPool(Pool pool) { this.pool = pool; } public static class Pool extends GenericObjectPoolConfig<ChannelSftp> { private int maxTotal = DEFAULT_MAX_TOTAL; private int maxIdle = DEFAULT_MAX_IDLE; private int minIdle = DEFAULT_MIN_IDLE; public Pool() { super(); } @Override public int getMaxTotal() { return maxTotal; } @Override public void setMaxTotal(int maxTotal) { this.maxTotal = maxTotal; } @Override public int getMaxIdle() { return maxIdle; } @Override public void setMaxIdle(int maxIdle) { this.maxIdle = maxIdle; } @Override public int getMinIdle() { return minIdle; } @Override public void setMinIdle(int minIdle) { this.minIdle = minIdle; } } }
Pool
SftpFactory.java
package com.vipsoft.sftp.pool; import com.jcraft.jsch.ChannelSftp; import com.jcraft.jsch.JSch; import com.jcraft.jsch.JSchException; import com.jcraft.jsch.Session; import com.vipsoft.sftp.config.SftpProperties; import com.vipsoft.sftp.exception.SftpPoolException; import org.apache.commons.pool2.BasePooledObjectFactory; import org.apache.commons.pool2.PooledObject; import org.apache.commons.pool2.impl.DefaultPooledObject; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import java.util.Properties; public class SftpFactory extends BasePooledObjectFactory<ChannelSftp> { private final Logger logger = LoggerFactory.getLogger(this.getClass()); private SftpProperties properties; public SftpProperties getProperties() { return properties; } public void setProperties(SftpProperties properties) { this.properties = properties; } public SftpFactory(SftpProperties properties) { this.properties = properties; } @Override public ChannelSftp create() { try { JSch jsch = new JSch(); Session sshSession = jsch.getSession(properties.getUsername(), properties.getHost(), properties.getPort()); sshSession.setPassword(properties.getPassword()); Properties sshConfig = new Properties(); sshConfig.put("StrictHostKeyChecking", "no"); sshSession.setConfig(sshConfig); sshSession.connect(); ChannelSftp channel = (ChannelSftp) sshSession.openChannel("sftp"); channel.connect(); return channel; } catch (JSchException e) { throw new SftpPoolException("连接sfpt失败", e); } } @Override public PooledObject<ChannelSftp> wrap(ChannelSftp channelSftp) { return new DefaultPooledObject<>(channelSftp); } // 销毁对象 @Override public void destroyObject(PooledObject<ChannelSftp> p) { ChannelSftp channelSftp = p.getObject(); channelSftp.disconnect(); } }
SftpPool.java
package com.vipsoft.sftp.pool; import com.jcraft.jsch.ChannelSftp; import org.apache.commons.pool2.impl.GenericObjectPool; public class SftpPool<T> extends GenericObjectPool<ChannelSftp> { public SftpPool(SftpFactory factory) { super(factory,factory.getProperties().getPool()); } /** * 获取一个sftp连接对象 * @return sftp连接对象 */ @Override public ChannelSftp borrowObject() throws Exception { return super.borrowObject(); } /** * 归还一个sftp连接对象 * @param channelSftp sftp连接对象 */ @Override public void returnObject(ChannelSftp channelSftp) { if (channelSftp!=null) { super.returnObject(channelSftp); } } }
Utils
ByteUtil.java
package com.vipsoft.sftp.utils; import com.jcraft.jsch.ChannelSftp; import com.jcraft.jsch.SftpException; import com.vipsoft.sftp.exception.SftpPoolException; import com.vipsoft.sftp.pool.SftpPool; import java.io.InputStream; public class SftpUtil { private SftpPool pool; public SftpUtil(SftpPool pool) { this.pool = pool; } /** * 下载文件 * * @param dir 远程目录 * @param name 远程文件名 * @return 文件字节数组 */ public byte[] download(String dir, String name) { ChannelSftp sftp = null; try { sftp = pool.borrowObject(); sftp.cd(dir); InputStream in = sftp.get(name); return ByteUtil.inputStreamToByteArray(in); } catch (Exception e) { throw new SftpPoolException("sftp下载文件出错", e); } finally { pool.returnObject(sftp); } } /** * 上传文件 * * @param dir 远程目录 * @param name 远程文件名 * @param in 输入流 */ public void upload(String dir, String name, InputStream in) { ChannelSftp sftp = null; try { sftp = pool.borrowObject(); mkdirs(sftp, dir); sftp.cd(dir); sftp.put(in, name); } catch (Exception e) { throw new SftpPoolException("sftp上传文件出错", e); } finally { pool.returnObject(sftp); } } /** * 删除文件 * * @param dir 远程目录 * @param name 远程文件名 */ public void delete(String dir, String name) { ChannelSftp sftp = null; try { sftp = pool.borrowObject(); sftp.cd(dir); sftp.rm(name); } catch (Exception e) { throw new SftpPoolException("sftp删除文件出错", e); } finally { pool.returnObject(sftp); } } /** * 递归创建多级目录 * * @param dir 多级目录 */ private void mkdirs(ChannelSftp sftp, String dir) { String[] folders = dir.split("/"); try { sftp.cd("/"); for (String folder : folders) { if (folder.length() > 0) { try { sftp.cd(folder); } catch (Exception e) { sftp.mkdir(folder); sftp.cd(folder); } } } } catch (SftpException e) { throw new SftpPoolException("sftp创建目录出错", e); } } }
Test
SftpTest.java
package com.vipsoft.sftp; import com.vipsoft.sftp.utils.SftpUtil; import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; @SpringBootTest public class SftpTest { @Autowired private SftpUtil sftpUtil; @Test void downloadTest() { byte[] dockerfiles = sftpUtil.download("/opt/demo/", "Dockerfile"); System.out.println("FileSize =>" + dockerfiles.length); } }
到此这篇关于Java GenericObjectPool 对象池化技术--SpringBoot sftp 连接池工具类的文章就介绍到这了,更多相关Java 对象池GenericObjectPool内容请搜索脚本之家以前的文章或继续浏览下面的相关文章希望大家以后多多支持脚本之家!