Java对接ansible自动运维化平台方式
作者:猛男的温柔
Java对接ansible自动运维化平台实现文件采集分发
经过大量查阅,网上使用Java对接ansible自动运维化平台的示例代码几乎没有,为了方便自己后期巩固以及有需要的小伙伴,特以记录!!!
此次对接主要为以下两个功能:
- 文件采集(对文件进行批量操作,包括批量从多台主机中采集共性文件如日志文件)
- 文件分发(对文件进行批量操作,包括批量从多台主机中分发共性文件如日志文件)
场景说明及ansible yum安装
因ansible没有Windows的安装包,所以为了方便测试,搭建了一套Linux环境进行后续工作。
此次采用yum方式安装,在采用yum方式安装Ansible,首先安装EPEL源。
yum install -y http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
查看EPEL源中的Ansible版本
yum info ansible
直接安装此版本,如果有其他要求,请调整源,安装其他ansible版本
yum install -y ansible
安装完成之后,查看ansible版本信息
ansible --version
配置Ansible服务器清单
清单文件/etc/ansible/hosts,在此文件中编写节点主机的对应IP地址和端口
我这里只是做一个演示,其中IP后面可以添加节点真实的SSH的端口,在定义的内容上面有一个[]列表,里面的内容为自定义内容,方面为了操作绑定的节点主机,我习惯称之为分组列表
简单的认证一下,Ping一下添加的主机
成功安装ansible !!
Java代码实现文件分发
顾名思义,文件分发就是把本机的文件分发到多个主机。
这时候就需要 Apache POI(大家可以去导入对应的包)来创建本机的文件了(ansible Host配置文件也通过POI创建)
POI创建文件工具类
package com.tiduyun.cmp.operation.utils; import com.tiduyun.cmp.common.model.operation.HostInfo; import lombok.extern.slf4j.Slf4j; import org.apache.commons.lang.StringUtils; import org.springframework.stereotype.Component; import java.io.File; import java.io.FileOutputStream; import java.io.IOException; import java.util.ArrayList; import java.util.List; /** * @author huyuan@tiduyun.com ansible创建文件 */ @Slf4j @Component public class AnsibleCreateFileUtils { private final static String filename = "hosts"; public static String passWordConnect(List<HostInfo> hostInfo, String hostGroup , String directory) throws IOException{ /** 在本地新建一个文件夹 里面创建一个文件 向里面写入内容 */ // 创建文件夹对象 创建文件对象 File folder = new File(directory); // 如果文件夹不存在 就创建一个空的文件夹 if (!folder.exists()) { log.info("创建了文件夹{}" , folder); folder.mkdirs(); } File file = new File(directory, filename); // 如果文件不存在 就创建一个空的文件 if (!file.exists()) { try { log.info("创建了文件{}" , file); file.createNewFile(); } catch (IOException e) { log.error("error data{}" , e); } } // 写入数据 // 创建文件字节输出流 FileOutputStream fos = new FileOutputStream(file); try { List<String> list = new ArrayList<>(); for (HostInfo data : hostInfo) { // 开始写 String string = data.getHost() + " ansible_ssh_pass=" + data.getPasswd() + " ansible_ssh_user=" + data.getAccount() + " ansible_ssh_port=" + data.getPort(); list.add(string); } String splicingData = StringUtils.join(list, "\n"); String str = "[" + hostGroup + "]" + "\n" + splicingData; byte[] bytes = str.getBytes(); // 将byte数组中的所有数据全部写入 fos.write(bytes); fos.flush(); log.info("文件内容{}" , str); // 删除文件 // deleteFile(file); // 关闭流 } catch (IOException e) { log.error("error data{}" , e); throw e; }finally { if (fos != null) { fos.close(); } } return directory; } public static void deleteFile(File file) { if (file.exists()) {// 判断路径是否存在 if (file.isFile()) {// boolean isFile():测试此抽象路径名表示的文件是否是一个标准文件。 file.delete(); } else {// 不是文件,对于文件夹的操作 // 保存 路径D:/1/新建文件夹2 下的所有的文件和文件夹到listFiles数组中 File[] listFiles = file.listFiles();// listFiles方法:返回file路径下所有文件和文件夹的绝对路径 for (File file2 : listFiles) { /* * 递归作用:由外到内先一层一层删除里面的文件 再从最内层 反过来删除文件夹 * 注意:此时的文件夹在上一步的操作之后,里面的文件内容已全部删除 * 所以每一层的文件夹都是空的 ==》最后就可以直接删除了 */ deleteFile(file2); } } file.delete(); } else { log.error("该file路径不存在!!"); } } }
创建主机组配置文件
注:ansible分为两种连接方式,这里采用的是密钥连接,生成的文件已拼接密钥!!!后续的采集与分发都要用到这个。(如有不懂的小伙伴,可以去查找一下ansible的连接方式)
@Override public void ansibleCreateHost(HostInfo hostInfo, String Key) { ParamCheckUtils.notNull(hostInfo, "hostInfo"); List<HostInfo> HostIp = Arrays.asList(hostInfo); for (HostInfo data : HostIp) { String ansiblePassWd = data.getPasswd(); String PassWd = hostInfoService.decode(ansiblePassWd); data.setPasswd(PassWd); } try { AnsibleCreateFileUtils.passWordConnect(HostIp, ansibleConfigurationItemVo.getHostGroup(), ansibleConfigurationItemVo.getDirectory()); } catch (IOException e) { log.error("Failed to create host configuration{}", e); } }
实现文件分发
主机配置文件已经配置好,接下来就是执行ansible对应的命令,通过Java拼接ansible命令。
执行命令工具类
package com.tiduyun.cmp.operation.utils; import lombok.extern.slf4j.Slf4j; import java.io.BufferedReader; import java.io.IOException; import java.io.InputStreamReader; import java.nio.charset.Charset; import static cn.hutool.db.DbUtil.close; /** * @author huyuan@tiduyun.com ansible执行命令工具类 * upload 上传文件 * createRemoteDirectory 创建远程目录 */ @Slf4j public class AnsibleExecuteTheOrderUtils { private final static String commandBin = "/bin/sh"; private final static String commandC = "-c"; /** * 创建远程目录 */ public static void createRemoteDirectory(String hostGroup, String remotePath, String directory) throws IOException { Runtime run = Runtime.getRuntime(); String[] cmds = new String[3]; cmds[0] = commandBin; cmds[1] = commandC; cmds[2] = "ansible " + hostGroup + " -m command -a " + "\"mkdir " + remotePath + "\"" + " -i " + directory + "/hosts"; // 执行CMD命令 Process p = run.exec(cmds); log.info("ansible远程执行命令为{}", cmds[2]); BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream(), Charset.forName("UTF-8"))); try { String lineMes; while ((lineMes = br.readLine()) != null) log.info(lineMes);// 打印输出信息 try { // 检查命令是否执行失败。 if (p.waitFor() != 0) { if (p.exitValue() == 1)// 0表示正常结束,1:非正常结束 log.error("命令执行失败"); } } catch (InterruptedException e) { log.error("error data{}", e); } } catch (IOException e) { log.error("fail to carry out command{}", e); throw e; } finally { if (br != null) { br.close(); } } } /** * 文件分发 */ public static void upload(String hostGroup, String localPath, String remotePath, String directory) throws IOException { Runtime run = Runtime.getRuntime(); String[] cmds = new String[3]; cmds[0] = commandBin; cmds[1] = commandC; cmds[2] = "ansible " + hostGroup + " -m copy -a " + "\"src=" + localPath + " dest=" + remotePath + "\"" + " -i " + directory + "/hosts"; // 执行CMD命令 Process p = run.exec(cmds); log.info("ansible命令为{}", cmds[2]); BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream(), Charset.forName("UTF-8"))); try { String lineMes; while ((lineMes = br.readLine()) != null) log.info("ansible输出信息为 :" + lineMes);// 打印输出信息 try { // 检查命令是否执行失败。 if (p.waitFor() != 0) { if (p.exitValue() == 1)// 0表示正常结束,1:非正常结束 log.error("命令执行失败"); } } catch (InterruptedException e) { log.error("error data{}", e); } } catch (IOException e) { log.error("fail to carry out command{}", e); throw e; } finally { if (br != null) { br.close(); } } } /** * 文件采集 */ public static void fileCollection(String hostGroup, String remotePath, String localPath , String directory) throws IOException { Runtime run = Runtime.getRuntime(); String[] cmds = new String[3]; cmds[0] = commandBin; cmds[1] = commandC; cmds[2] = "ansible " + hostGroup + " -m fetch -a " + "\"src=" + remotePath + " dest=" + localPath + " force=yes backup=yes\"" + " -i " + directory + "/hosts"; // 执行CMD命令 Process p = run.exec(cmds); log.info("ansible远程采集文件命令为{}", cmds[2]); BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream(), Charset.forName("UTF-8"))); try { String lineMes; while ((lineMes = br.readLine()) != null) log.info(lineMes);// 打印输出信息 try { // 检查命令是否执行失败。 if (p.waitFor() != 0) { if (p.exitValue() == 1)// 0表示正常结束,1:非正常结束 log.error("命令执行失败"); } } catch (InterruptedException e) { log.error("error data{}", e); } } catch (IOException e) { log.error("fail to carry out command{}", e); throw e; } finally { if (br != null) { br.close(); } } } public static void ExecuteTheOrder(String command) throws IOException { log.info("start execute cmd {}", command); String[] cmd = new String[] {"/bin/bash", "-c", command}; Runtime run = Runtime.getRuntime(); Process p = run.exec(cmd); // 执行CMD命令 BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream(), Charset.forName("UTF-8"))); try { String lineMes; while ((lineMes = br.readLine()) != null) log.info("输出信息为 {}", lineMes);// 打印输出信息 try { // 检查命令是否执行失败。 if (p.waitFor() != 0) { if (p.exitValue() == 1)// 0表示正常结束,1:非正常结束 log.error("命令执行失败"); } } catch (InterruptedException e) { log.error("error data{}", e); } } catch (IOException e) { log.error("fail to carry out command{}", e); throw e; } finally { if (br != null) { br.close(); } } } public static void disconnect() { try { close(); } catch (Exception ex) { // Ignore because disconnection is quietly } } // public void execute(String command) throws Exception { // log.info("start execute cmd {}", command); // try (Session session = sshClient.startSession()) { // Session.Command exec = session.exec(command); // // Integer readLineCount = 0; // InputStream in = exec.getInputStream(); // log.info(IOUtils.readFully(in).toString()); // String errorMessage = IOUtils.readFully(exec.getErrorStream(), LoggerFactory.DEFAULT).toString(); // log.info(errorMessage); // if (exec.getExitStatus() != null && exec.getExitStatus() != 0) { // throw new RuntimeException( // "exec " + command + " error,error message is " + errorMessage + ",error code " + exec.getExitStatus()); // } // log.info("exec result code {}", exec.getExitStatus()); // // } // // } }
接下来就是调用
package com.tiduyun.cmp.operation.service.impl; import cn.hutool.core.util.RandomUtil; import cn.hutool.core.util.StrUtil; import com.tiduyun.cmp.common.model.flow.UploadFile; import com.tiduyun.cmp.common.model.operation.ComponentInfo; import com.tiduyun.cmp.common.model.operation.HostInfo; import com.tiduyun.cmp.common.provider.service.ExceptionBuildService; import com.tiduyun.cmp.operation.constant.OperationExceptionCode; import com.tiduyun.cmp.operation.constant.StartCmdSeparate; import com.tiduyun.cmp.operation.model.AnsibleConfigurationItemVo; import com.tiduyun.cmp.operation.model.vo.FileQueryVo; import com.tiduyun.cmp.operation.service.AnsibleTaskRecordService; import com.tiduyun.cmp.operation.service.ComposerDeployService; import com.tiduyun.cmp.operation.service.HostInfoService; import com.tiduyun.cmp.operation.service.UploadFileService; import com.tiduyun.cmp.operation.utils.AnsibleExecuteTheOrderUtils; import lombok.extern.slf4j.Slf4j; import org.apache.commons.collections4.CollectionUtils; import org.apache.commons.lang3.StringUtils; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Value; import org.springframework.stereotype.Service; import java.io.File; import java.util.ArrayList; import java.util.Arrays; import java.util.List; @Slf4j @Service public class AnsibleDeployServiceImpl implements ComposerDeployService { @Value(value = "${cmp.operation.commandHeader:cmd /c}") private String commandHeader; @Value(value = "${cmp.operation.filePath:/data/cmp/file}") private String filePath; @Value(value = "${cmp.operation.remoteFilePath:/tmp}") private String remoteFilePath; @Autowired private AnsibleTaskRecordService ansibleTaskRecordService; @Autowired private AnsibleConfigurationItemVo ansibleConfigurationItemVo; @Autowired private UploadFileService uploadFileService; @Autowired private HostInfoService hostInfoService; @Autowired private ExceptionBuildService exceptionBuildService; @Override public void deploy(HostInfo hostInfo, ComponentInfo componentInfo, String cpmposerName) { ansibleTaskRecordService.ansibleCreateHost(hostInfo, null); try { String remotePath = StringUtils.join(remoteFilePath, "/", cpmposerName, "-", componentInfo.getName(), "-", RandomUtil.randomString(3)); log.info("remote file path = {}", remotePath); List<Integer> fileIds = getFileIds(componentInfo.getFileUrl()); if (CollectionUtils.isNotEmpty(fileIds)) { FileQueryVo uploadFileQueryVo = new FileQueryVo(); uploadFileQueryVo.setIds(fileIds); List<UploadFile> uploadFiles = uploadFileService.query(uploadFileQueryVo); for (UploadFile uploadFile : uploadFiles) { String path = StringUtils.join(filePath, uploadFile.getFilePath()); File file = new File(path); if (!file.exists()) { log.error("file url is {}", file.getPath()); throw exceptionBuildService.buildException(OperationExceptionCode.FILE_NOT_EXIST, new Object[] {uploadFile.getFileName()}); } // 创建远程目录 AnsibleExecuteTheOrderUtils.createRemoteDirectory(ansibleConfigurationItemVo.getHostGroup(), StringUtils.join(remotePath), ansibleConfigurationItemVo.getDirectory()); // 分发文件 AnsibleExecuteTheOrderUtils.upload(ansibleConfigurationItemVo.getHostGroup(), path, StringUtils.join(remotePath, "/", uploadFile.getFileName()), ansibleConfigurationItemVo.getDirectory()); } } List<String> startCmds = getStartCmds(componentInfo.getStartCmd()); if (CollectionUtils.isNotEmpty(startCmds)) { String cdCmd = StringUtils.join("cd ", remotePath); String execCmd = StringUtils.join(startCmds, ";"); execCmd = StringUtils.join(cdCmd, ";", execCmd); log.info("execCmd= " + execCmd); // sshClient.execute(execCmd); AnsibleExecuteTheOrderUtils.ExecuteTheOrder(execCmd); } else { log.error("parse startCmd fail {}", componentInfo.getStartCmd()); } } catch (Exception e) { log.error("主机[{}]部署[{}]组件失败,主机ID[{}],组件ID[{}]:", hostInfo.getHost(), componentInfo.getName(), hostInfo.getId(), componentInfo.getId(), e); throw exceptionBuildService.buildException(OperationExceptionCode.EXECUTE_CMD_ERROR, new Object[] {e.getMessage()}); } finally { AnsibleExecuteTheOrderUtils.disconnect(); } } @Override public boolean isSupport(HostInfo hostInfo) { return true; } private List<Integer> getFileIds(String fileIds) { List<Integer> ids = new ArrayList<>(); if (fileIds == null) { return null; } String[] split = StringUtils.split(fileIds, ","); for (String s : split) { ids.add(Integer.parseInt(s)); } return ids; } private List<String> getStartCmds(String startCmd) { List<String> cmd = new ArrayList<>(); if (startCmd == null) { return cmd; } String[] split = StrUtil.split(startCmd, StartCmdSeparate.SIGN); cmd.addAll(Arrays.asList(split)); return cmd; } public static Boolean needCd(String s) { String[] splits = StrUtil.split(s, "&&"); int maxIndex = splits.length - 1; String cmd = splits[maxIndex]; if (StrUtil.startWith(cmd, "cd")) { return false; } else { return true; } } }
文件采集
同上,调用两个工具类
@Override public void fileCollection(HostInfo hostInfo, String remotePath, String localPath) { ansibleCreateHost(hostInfo, null); try { log.info("remote file path = {}", remotePath); log.info("local file path = {}", localPath); // 文件采集 AnsibleExecuteTheOrderUtils.fileCollection(ansibleConfigurationItemVo.getHostGroup(), remotePath, localPath , ansibleConfigurationItemVo.getDirectory()); } catch (Exception e) { log.error("主机[{}]文件采集失败,主机ID[{}]:", hostInfo.getHost(), hostInfo.getId(), e); throw exceptionBuildService.buildException(OperationExceptionCode.EXECUTE_CMD_ERROR, new Object[] {e.getMessage()}); } finally { AnsibleExecuteTheOrderUtils.disconnect(); } }
总结
以上代码如大家有需要,请自行更改!!!
这些仅为个人经验,希望能给大家一个参考,也希望大家多多支持脚本之家。