Java Api实现Elasticsearch的滚动查询功能
作者:Cimbala
这篇文章主要介绍了Java Api实现Elasticsearch的滚动查询,解决ES每次只能查询一万条数据的问题,本文通过实例代码给大家介绍的非常详细,需要的朋友可以参考下
解决ES每次只能查询一万条数据的问题
@Override public List<ESHandleDto> getVisitorsNum(String startTime, String endTime, String schoolName, String typeFunction) throws IOException { List<ESHandleDto> esHandleDtos = new ArrayList<>(); SearchRequest searchRequest = new SearchRequest(); searchRequest.indices(ElasticEnum.FUNCTIONLOG_INDEX.getValue()); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery(); if (StringUtils.hasText(schoolName)) { boolQueryBuilder.must(QueryBuilders.termQuery("schoolName.keyword", schoolName)); } if (StringUtils.hasText(typeFunction)) { boolQueryBuilder.must(QueryBuilders.termQuery("typeFunction.keyword", typeFunction)); } if (StringUtils.hasText(startTime) && StringUtils.hasText(endTime)) { boolQueryBuilder.must(QueryBuilders.rangeQuery("createDate").gte(startTime + "T00:00:00.000Z").lte(endTime + "T23:59:59.000Z")); } BoolQueryBuilder shouldQuery = QueryBuilders.boolQuery(); shouldQuery.should().add(QueryBuilders.termQuery("description.keyword", "查询学生信息表")); shouldQuery.should().add(QueryBuilders.termQuery("description.keyword", "获取学校访客数据")); boolQueryBuilder.must(shouldQuery); sourceBuilder.sort("_doc", SortOrder.DESC); sourceBuilder.size(10000); sourceBuilder.query(boolQueryBuilder); searchRequest.scroll(TimeValue.timeValueMinutes(1)); searchRequest.source(sourceBuilder); // System.out.println(sourceBuilder.toString()); SearchResponse searchResponse = null; try { searchResponse = esConfig.restHighLevelClient().search(searchRequest, RequestOptions.DEFAULT); } catch (Throwable e) { throw new RuntimeException(e); } scrollHandle(searchResponse, esConfig, esHandleDtos, startTime, endTime); return esHandleDtos; } public static void scrollHandle(SearchResponse searchResponse, ESConfig esConfig, List<ESHandleDto> esHandleDtos, String startTime, String endTime) { SearchHits hits = searchResponse.getHits(); String scrollId = searchResponse.getScrollId(); SearchHit[] searchHits = hits.getHits(); //对结果集处理 List<FunctionLogElasticEntity> functionLogs = ESUtil.convertToFunctionLog(searchHits); visitorsResultHandle(functionLogs, esHandleDtos, ESUtil.isMoreThanMonth(startTime, endTime)); //滚动查询部分,将从第10001笔数据开始 while (searchHits != null && searchHits.length > 0) { SearchScrollRequest searchScrollRequest = new SearchScrollRequest(scrollId); searchScrollRequest.scroll(TimeValue.timeValueMinutes(1)); try { searchResponse = esConfig.restHighLevelClient().scroll(searchScrollRequest, RequestOptions.DEFAULT); } catch (Throwable e) { throw new RuntimeException(e); } scrollId = searchResponse.getScrollId(); hits = searchResponse.getHits(); searchHits = hits.getHits(); //对结果集处理 functionLogs = ESUtil.convertToFunctionLog(searchHits); visitorsResultHandle(functionLogs, esHandleDtos, ESUtil.isMoreThanMonth(startTime, endTime)); } //清除滚动,否则影响下次查询 ClearScrollRequest clearScrollRequest = new ClearScrollRequest(); clearScrollRequest.addScrollId(scrollId); ClearScrollResponse clearScrollResponse = null; try { clearScrollResponse = esConfig.restHighLevelClient().clearScroll(clearScrollRequest, RequestOptions.DEFAULT); } catch (IOException e) { throw new RuntimeException(e); } boolean succeeded = clearScrollResponse.isSucceeded(); System.out.println(succeeded); }
到此这篇关于Java Api实现Elasticsearch的滚动查询的文章就介绍到这了,更多相关Java Elasticsearch滚动查询内容请搜索脚本之家以前的文章或继续浏览下面的相关文章希望大家以后多多支持脚本之家!
您可能感兴趣的文章:
- Java查询Elasticsearch数据根据指定id检索(in查询)、sql权限过滤、多字段匹配检索及数据排序
- Java使用ES Client 调用滚动查询及Elasticsearch滚动查询Scrolling机制
- Java利用ElasticSearch实现自动补全功能
- Java+ElasticSearch+Pytorch实现以图搜图功能
- Java中Elasticsearch 实现分页方式(三种方式)
- 关于Java中配置ElasticSearch集群环境账号密码的问题
- Java如何使用elasticsearch进行模糊查询
- JAVA使用ElasticSearch查询in和not in的实现方式
- JAVA操作elastic search的详细过程