제가 약 0.1기가정도 되는 text파일을 split하여 mysql에 multi thread로 넣습니다. <div>대충 순서가 </div> <div>BufferedReader로 라인별로 읽고 => ArrayBlockingQueue에 넣고 (multi thread로 여러파일을 읽으면서 이 queue에 다 넣습니다.) queue의 사이즈가</div> <div>일정량이되면 => StringBuilder에 append하고 => splite작업을 거치고 Service=> DAO로 가서 insert하는 중에</div> <div><font color="#ff0000">Deadlock found when trying to get lock; try restarting transaction</font> 에러에 걸립니다.</div> <div>service의 insert메서드에 synchronized를 적용했는데도 에러가 걸리네요 db에 넣는데 시간이 오래 걸려서 속도도 안나올 뿐 더러</div> <div>Batch모드로 돌리면 빠르다고 하는데 </div> <div> Mapper에서 useGeneratedKeys="true"<span style="font-size:9pt;white-space:pre;"> </span><span style="font-size:9pt;">keyProperty="contents_seq" 이것을 쓰고 있어서 사용도 불가능 합니다.</span></div> <div><span style="font-size:9pt;">혹시 조언 좀 해주실 수 있으신가요?? 작은 것이라도 괜찮습니다.</span></div> <div><br></div> <div>//lbqueue는 arrayBlockingQueue입니다.</div> <div><br></div> <div><div>@Override</div> <div><span style="white-space:pre;"> </span>public void run(){</div> <div><span style="white-space:pre;"> </span>String line = "";</div> <div><span style="white-space:pre;"> </span>try {</div> <div><span style="white-space:pre;"> </span>while ((line = in.readLine()) != null) {</div> <div><span style="white-space:pre;"> </span>lbqueue.offer(line + "\r\n");</div> <div><span style="white-space:pre;"> </span>if (lbqueue.size() > 0) {</div> <div><span style="white-space:pre;"> </span>if (line.indexOf("\r\n|\r|\n|\n\r") != -1) {</div> <div><span style="white-space:pre;"> </span>lbqueue.offer("\r\n");</div> <div><span style="white-space:pre;"> </span>}</div> <div><span style="white-space:pre;"> </span>}</div> <div><span style="white-space:pre;"> </span>if(lbqueue.size()>2500){</div> <div><span style="white-space:pre;"> </span>String s="";</div> <div><span style="white-space:pre;"> </span>StringBuilder builder= new StringBuilder();</div> <div><span style="white-space:pre;"> </span>try {</div> <div><span style="white-space:pre;"> </span>while(lbqueue.size()>0){</div> <div><span style="white-space:pre;"> </span>s=lbqueue.take();</div> <div><span style="white-space:pre;"> </span>if(s.equals(""))continue;</div> <div><span style="white-space:pre;"> </span>builder.append(s);</div> <div><span style="white-space:pre;"> </span>}</div> <div><span style="white-space:pre;"> </span>} catch (InterruptedException e) {</div> <div><span style="white-space:pre;"> </span>e.printStackTrace();</div> <div><span style="white-space:pre;"> </span>}</div> <div><span style="white-space:pre;"> </span>finally {</div> <div><span style="white-space:pre;"> </span>//service에 synchronized <span style="font-size:9pt;">insertMassURL</span>(ArrayList<HashMap<String, String>> arrayList) 이런 식으로 선<span style="white-space:pre;"> </span>언해놓았습니다</div> <div><span style="white-space:pre;"> </span>service.insertMassURL(csvTypeSpliter(builder.toString()));</div> <div><span style="white-space:pre;"> </span></div> <div><span style="white-space:pre;"> </span>}</div> <div><span style="white-space:pre;"> </span>}</div> <div><span style="white-space:pre;"> </span>}</div> <div><span style="white-space:pre;"> </span></div> <div><span style="white-space:pre;"> </span>//latch.countDown();</div> <div><span style="white-space:pre;"> </span></div> <div><span style="white-space:pre;"> </span>} catch (IOException e1) {</div> <div><span style="white-space:pre;"> </span>// TODO Auto-generated catch block</div> <div><span style="white-space:pre;"> </span>e1.printStackTrace();</div> <div><span style="white-space:pre;"> </span>}</div> <div><span style="white-space:pre;"> </span>finally {</div> <div><span style="white-space:pre;"> </span></div> <div><span style="white-space:pre;"> </span>}</div> <div><span style="white-space:pre;"> </span>//service.insertMassURL(htmlFileReader(builder.toString()));</div> <div><span style="white-space:pre;"> </span>}</div></div>