pgµç×Ó¹ÙÍøÔÆº£Insight HD,ÊÇÒ»Ì×¾¹ýµ÷Óź͹¦Ð§ÔöÇ¿µÄHadoopÓëSparkÆóÒµ¿¯Ðа棬°üÀ¨HadoopÉú̬ÖеÄ20+Ö÷Òª×é¼þ£¬×ÊÖú¿Í»§ÇáËÉÓ¦¶Ôº£Á¿Êý¾ÝµÄÊÕÂÞ¡¢´æ´¢¡¢ÅÌËã¡¢ÆÊÎöÍÚ¾òºÍÊý¾ÝÄþ¾²µÈÓ¦Óó¡¾°¡£
¡¡¡¡

¡¡¡¡¶àÔ´Êý¾ÝµÄ¸ßЧ¼¯³É
¡¡¡¡Ôƺ£Insight HDÌṩ¶àÔ´Êý¾Ý£¨°üÀ¨½á¹¹»¯¡¢°ë½á¹¹»¯ºÍ·Ç½á¹¹»¯Êý¾Ý£©µÄ¼¯³ÉÄÜÁ¦£¬Ìṩ¸ßÍÌÍ¡¢¿ÉÀ©Õ¹µÄÊý¾Ý×ÜÏߺÍÊý¾Ý·Ö·¢¹¦Ð§£¬Ö§³ÖÅúÁ¿¼ÓÔØ¡¢ÊµÊ±¼ÓÔØ¡¢Êý¾Ý¿â¼ÓÔØ¡¢Îļþ¼ÓÔØµÈ¶àÖÖ¼ÓÔØ·½·¨¡£
¡¡¡¡Òì¹¹Êý¾ÝµÄº£Á¿´æ´¢
¡¡¡¡Ôƺ£Insight HDÌṩ»ùÓÚÂþÑÜʽÎļþϵͳºÍ²¢Ðмܹ¹µÄ´óÊý¾Ý´æ´¢ÄÜÁ¦£¬Ö§³ÖPB¼¶Êý¾Ý¹æÄ£µÄ¸ß¿É¿¿ºÍ¸ß¿ÉÓô洢£¬Ö§³Ö´æ·Å¶àÖÖÎļþ»¨Ñù£¬ÀýÈç¹ØÏµÊý¾Ý¿âµÈ½á¹¹»¯Êý¾Ý£¬ÈÕÖ¾¡¢ÍøÒ³µÈ°ë½á¹¹»¯Êý¾Ý£¬ÒÔ¼°ÊÓÆµ¡¢Í¼Æ¬¡¢ÎĵµµÈ·Ç½á¹¹»¯Êý¾Ý¡£
¡¡¡¡¶à³¡¾°ÏµÄÊý¾ÝÅÌËã¿ò¼Ü
¡¡¡¡ÃæÏò²î±ðÒµÎñ³¡¾°£¬Ôƺ£Insight HDÌṩÀëÏßÅÌËã¡¢Á÷ʽÅÌËã¡¢ÄÚ´æÅÌË㡢ͼÅÌËãµÈ¸»ºñµÄÅÌËã¿ò¼Ü£¬Ö§³ÖÅÌËãÈÎÎñÁ÷³Ì±àÅÅ¡¢¼Æ»®°²ÅÅ£¬Ìṩ±ê×¼SQLµÄÊý¾Ý»á¼ûÄÜÁ¦¡£
¡¡¡¡»ùÓÚ»úеѧϰµÄ´óÊý¾ÝÆÊÎö
¡¡¡¡Ôƺ£Insight HDÌṩº¸Ç¶àÔ´Êý¾Ý½ÓÈë¡¢Êý¾ÝÌØÕ÷ÌáÈ¡¡¢Ë㷨ģÐÍÖÎÀí¡¢Ë㷨ģÐÍÆÀ¹ÀºÍ½á¹ûÔ¤²âµÈÍêÕû»úеѧϰÀú³ÌµÄ¿ÉÊÓ»¯´óÊý¾ÝÆÊÎö¹¦Ð§¡£Ö§³Ö¶àÔª·ÖÀà¡¢»Ø¹éÆÊÎö¡¢ÐÍ¬ÍÆ¼öƽÆÊÎöģʽ£¬SVM¡¢ÆÓËØ±´Ò¶Ë¹¡¢K-Means¡¢ÏßÐԻعéµÈ10+ÖÖËã·¨£¬Ö§³ÖÅúÁ¿Ô¤²âºÍʵʱԤ²â¹¦Ð§²¢ÌṩAPI¡£Ô¤²âÀú³Ì»ùÓÚÄÚ´æ½øÐеü´úʽÅÌË㣬²¢ÇÒÖ§³ÖÂþÑÜʽÅÌË㣬¿ÉÒÔÓ¦¶Ôº£Á¿Êý¾ÝÆÊÎö¡£
¡¡¡¡Äþ¾²ÓÐЧµÄ×ÊÔ´¸ôÀëÓë¹²Ïí
¡¡¡¡Ôƺ£Insight HDÌṩ¶Ô½á¹¹»¯¡¢·Ç½á¹¹»¯Êý¾ÝµÄ¶àÓû§×ÊÔ´ÖÎÀí£¬Âú×ã¶ÔÊý¾Ý×ÊÔ´µÄȨÏÞ¸ôÀë¡¢Äþ¾²ÊÚȨºÍ×ÊÔ´µ÷Àí¡£Ìṩ¶àÓû§µÄ¿ÉÊÓ»¯Êý¾Ý¿Õ¼äÖÎÀí¹¦Ð§£¬Ö§³ÖÊý¾Ý¿Õ¼ä¾Þϸ¡¢ÎļþÊý¡¢ÅÌËãÐÐÁеȵÄÅä¶îÉèÖã»Ö§³Ö¶àÓû§¼äÊý¾Ý×ÊÔ´µÄ¿ª·Å¶©ÔĺÍÖ¸¶¨¹²Ïí£¬¹²ÏíȨÏÞ¿É¿ØÖƵ½ÎļþĿ¼¡¢±í¼°ÁÐ×å¼¶£»Ö§³Ö¶ÔÊý¾Ý»á¼ûµÄÉó¼ÆºÍ¼à¿Ø¡£
¡¡¡¡Í³Ò»µÄ´óÊý¾Ýƽ̨ÔËάÖÎÀí
¡¡¡¡Ôƺ£Insight HDÌṩÕë¶ÔHDFS¡¢MapReduce¡¢Hive¡¢HBase¡¢ZooKeeper¡¢OozieºÍSparkµÈ20+×é¼þµÄ×Ô¶¯»¯×°Öð²ÅÅ£¬²¢ÎªÆ½Ì¨ÌṩÍêÉÆµÄÅäÖÃÖÎÀí¡¢¼à¿Ø¸æ¾¯µÈÄÜÁ¦¡£
¡¡¡¡pgµç×Ó¹ÙÍøÔÆº£Insight?HDÒÑͨ¹ýȨÍþÈí¼þ²âÆÀʵÑéÊҵIJâÊÔÈÏÖ¤¡£
¡¡¡¡
1.ÐÔÄÜÖ¸±ê
²âÊÔÇé¿ö£º8½Úµã¼¯Èº£»½Úµãµä·¶ÅäÖãºCPU:2*E5-2640£¬Äڴ棺256G£¬Ó²ÅÌ£ºSATAÅÌ
| ·ÖÀà |
Ö¸±êÏî |
¹æ¸ñ |
| MapReduceÐÔÄÜÖ¸±ê |
WordCount£ºÆ½¾ùÿ½Úµã´¦ÀíÄÜÁ¦ |
8GB/·ÖÖÓ |
| Terasort£ºÆ½¾ùÿ½Úµã´¦ÀíÄÜÁ¦ |
6.4GB/·ÖÖÓ |
|
| SparkÐÔÄÜÖ¸±ê |
WordCount£ºÆ½¾ùÿ½Úµã´¦ÀíÄÜÁ¦ |
30GB/·ÖÖÓ/Node |
| Terasort£ºÆ½¾ùÿ½Úµã´¦ÀíÄÜÁ¦ |
9GB/·ÖÖÓ/Node |
|
| SQL on Hadoop |
Aggregation£ºÆ½¾ùÿ½Úµã´¦ÀíÄÜÁ¦ |
8GB/·ÖÖÓ |
| Join£ºÆ½¾ùÿ½Úµã´¦ÀíÄÜÁ¦ |
4GB/·ÖÖÓ |
|
| HBase |
100%Ëæ»ú¶Á£ºÆ½¾ùÿ½Úµã¶ÁÈ¡¼Í¼ÌõÊý£¨Ã¿Ìõ¼Í¼1KB£©£¬ÏìӦʱ¼äСÓÚ50MS |
30,000 Records/s |
| 100%Ëæ»úд£ºÆ½¾ùÿ½ÚµãдÈë¼Í¼ÌõÊý£¨Ã¿Ìõ¼Í¼1KB£©£¬ÏìӦʱ¼äСÓÚ50MS |
40,000 Records/s |
|
| ˳ÐòɨÃ裺ƽ¾ùÿ½Úµãscan¼Í¼ÌõÊý£¨Ã¿Ìõ¼Í¼1KB£©£¬ÏìӦʱ¼äСÓÚ50MS |
15,000 Records/s |
|
| 50%Ëæ»úд+50%Ëæ»ú¶Á£ºÆ½¾ùÿ½ÚµãдÈë¼Í¼ÌõÊý£¨Ã¿Ìõ¼Í¼1KB£©£¬ÏìӦʱ¼äСÓÚ50MS |
д25,000 Records/s ¶Á18,000 Records/s |
|
| 50%Ëæ»úд+50%ɨÃè¶Á£ºÆ½¾ùÿ½ÚµãдÈë¼Í¼ÌõÊý£¨Ã¿Ìõ¼Í¼1KB£©£¬ÏìӦʱ¼äСÓÚ50MS |
д30,000 Records/s ¶Á10,000 Records/s |
| ÐòºÅ |
³§ÉÌ |
CPU ϵÁÐ |
¼Ü¹¹ |
| 1 |
Intel |
ÖÁǿϵÁУ¬Xeon5X¡¢Xeon6XµÈ |
X86 |
| 2 |
Èȳ± |
FT-1500A¡¢FT-2000+¡¢FT-2500 |
ARM |
| 3 |
»ªÎª |
öïÅôϵÁÐ |
ARM |
2.2 ²Ù×÷ϵͳ¼æÈÝÐÔ
| ÐòºÅ |
²Ù×÷ϵͳ |
| 1 |
CentOS |
| 2 |
Redhat |
| 3 |
ÒøºÓ÷è÷ë |
| 4 |
Öбê÷è÷ë |
| 5 |
֏֑OS |
| 6 |
EulerOS |