Download: Hive Releases 进入下载目录

1
2
$ tar -xzvf hive-x.y.z.tar.gz
cp hive-default.xml.template hive-site.xml

设置环境变量

1
2
3
$ export HIVE_HOME=/path/to/hive/home
$ export PATH=$HIVE_HOME/bin:$PATH #
$ export METASTORE_PORT=9083

Running Hive Hive uses Hadoop, so: you must have Hadoop in your path OR

1
export HADOOP_HOME=<hadoop-install-dir>

In addition, you must create /tmp and /user/hive/warehouse (aka hive.metastore.warehouse.dir) and set them chmod g+w in HDFS before you can create a table in Hive. Commands to perform this setup:

1
2
3
4
$ $HADOOP_HOME/bin/hadoop fs -mkdir       /tmp
$ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w /tmp
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w /user/hive/warehouse

Running HCatalog To run the HCatalog server from the shell in Hive release 0.11.0 and later:

1
$ $HIVE_HOME/hcatalog/sbin/hcat_server.sh

To use the HCatalog command line interface (CLI) in Hive release 0.11.0 and later:

1
$ $HIVE_HOME/hcatalog/bin/hcat

For more information, see HCatalog Installation from Tarball and HCatalog CLI in the HCatalog manual. Running WebHCat (Templeton) To run the WebHCat server from the shell in Hive release 0.11.0 and later:

1
$ $HIVE_HOME/hcatalog/sbin/webhcat_server.sh

For more information, see WebHCat Installation in the WebHCat manual. DDL Operations https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DDLOperations 启动服务metastore $HIVE_HOME/bin/hive –service metastore & 启动hiveserver2 $HIVE_HOME/bin/hive –service hiveserver2 HiveClient: https://cwiki.apache.org/confluence/display/Hive/HiveClient last edited by Lefty Leverenz on Aug 20, 2014. 由于我用的hadoop2.5.0 hive 0.13.1 所以文档上有些内容不符合。这里贴上经过修改的python 代码,代码里的hive路径和数据文件路径根据情况修改

#!/usr/bin/env python

import sys
sys.path.append('/usr/local/share/applications/apache-hive-0.13.1-bin/apache-hive-0.13.1-bin/lib/py')
from hive_service import ThriftHive
from hive_service.ttypes import HiveServerException
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol

try:
    transport = TSocket.TSocket('localhost', 10000)
    transport = TTransport.TBufferedTransport(transport)
    protocol = TBinaryProtocol.TBinaryProtocol(transport)

    client = ThriftHive.Client(protocol)
    transport.open()

    client.execute("CREATE TABLE IF NOT EXISTS r(a STRING, b INT, c DOUBLE)")
    client.execute("LOAD DATA LOCAL INPATH '/usr/local/share/applications/apache-hive-0.13.1-bin/apache-hive-0.13.1-bin/ml-100k/u.data' OVERWRITE  INTO TABLE r")
    client.execute("SELECT * FROM r")
    while (1):
      row = client.fetchOne()
      if (row == None):
        break
      print row
    client.execute("SELECT * FROM r")
    print client.fetchAll()

    transport.close()

except Thrift.TException, tx:
    print '%s' % (tx.message)

Platform:Fedora release 19 (Schrödinger’s Cat) 3.14.13-100.fc19.x86_64 Installing software:

$ sudo yum install ssh
$ sudo yum install rsync

Dowload hadoop: Apache Download Mirrors 准备工作: 进入hadoop目录

$ vi etc/hadoop/hadoop-env.sh

set to the root of your Java installation

export JAVA_HOME=/path/to/java/home

Assuming your installation directory is /usr/local/hadoop

export HADOOP_PREFIX=/path/to/hadoop/home

伪分布式配置: 进入hadoop目录,编辑文件 etc/hadoop/core-site.xml:

fs.defaultFS hdfs://localhost.localdomain:9000

这里由于fedora 的/etc/hosts 默认配置localhost.localdomain 解析到127.0.0.1,所以跟官方文档有点不一样。 etc/hadoop/hdfs-site.xml:

dfs.replication 1

设置ssh key

$ ssh-keygen -t dsa -P ‘’ -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

按照官方文档设置之前检查/etc/ssh/sshd_config # The default requires explicit activation of protocol 1 Protocol 2 #此处去掉注释 #添加 HostKey ~/.ssh/id_dsa 检查AuthorizedKeysFile配置 # The default is to check both .ssh/authorized_keys and .ssh/authorized_keys2 # but this is overridden so installations will only check .ssh/authorized_keys AuthorizedKeysFile .ssh/authorized_keys 配置开机启动ssh服务

$ systemctl enable sshd.service

启动ssh服务

$ service sshd start

开放22端口

$ iptables -A INPUT -p tcp –dport 22 -j ACCEPT

[root@localhost apache-hive-0.13.1-bin]# $HIVE_HOME/hcatalog/sbin/hcat_server.sh start
Started metastore server init, testing if initialized correctly…
Metastore startup failed, see /usr/local/share/applications/apache-hive-0.13.1-bin/apache-hive-0.13.1-bin/hcatalog/sbin/../var/log/hcat.err
[root@localhost apache-hive-0.13.1-bin]# tail /usr/local/share/applications/apache-hive-0.13.1-bin/apache-hive-0.13.1-bin/hcatalog/sbin/../var/log/hcat.err
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:504)
at java.lang.Integer.parseInt(Integer.java:527)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HiveMetastoreCli.parse(HiveMetaStore.java:5025)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:5036)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

需要设置metastore 运行的端口,在环境变量里添加

export METASTORE_PORT=9083

今天查看博客情况发现get请求都没问题,wp-admin登录的时候出现502错误,怀疑是不是最近放到CDN上的问题,然后把CNAME又重新换成A记录,问题依然。随后查看wp-admin.php代码,显然从这上面是看不出有什么问题的,然后检查打开文件句柄数,根据http://hohoqi.blog.163.com/blog/static/1079556120110145464301/提供的以下内容检查了下,均是65535。 1、提升服务器的文件句柄打开打开 /etc/security/limits.conf : (增加) soft nofile 51200 hard nofile 51200 # vi /etc/security/limits.conf 加上 soft nofile 51200 hard nofile 51200 2、提升nginx的进程文件打开数 nginx.conf : worker_rlimit_nofile 51200; 3、修改php-fpm.conf文件,主要需要修改2处。 命令 ulimit -n 查看限制的打开文件数,php-fpm.conf 中的选项rlimit_files 确保和此数值一致。 10240 51200 4、 # vi /etc/sysctl.conf 底部添加 fs.file-max=51200 于是查看php-fpm.log和php-fpm.log.slow 两个日志均发现fpm_children_bury这样的warning,看来是执行时间太长,于是想起想起这两天装了个插件wordfence,一个安全插件,进不去后台禁用只好从ftp删除这个插件目录。问题解决!

Aug 29 11:44:32.175470 [NOTICE] fpm_children_make(), line 352: child 11729 (pool default) started
Aug 29 11:44:32.179330 [NOTICE] fpm_event_loop(), line 107: libevent: entering main loop
Aug 29 11:44:40.227249 [NOTICE] fpm_got_signal(), line 48: received SIGCHLD
Aug 29 11:44:40.227327 [WARNING] fpm_children_bury(), line 215: child 11718 (pool default) exited on signal 11 SIGSEGV after 8.060272 seconds from start
Aug 29 11:44:40.227856 [NOTICE] fpm_children_make(), line 352: child 11738 (pool default) started
Aug 29 11:46:59.065903 [WARNING] fpm_request_check_timed_out(), line 146: child 11725, script ‘/alidata/www/wordpress/wp-admin/admin-ajax.php’ (pool default) executing too slow (5.112693 sec), logging
Aug 29 11:46:59.066086 [NOTICE] fpm_got_signal(), line 48: received SIGCHLD
Aug 29 11:46:59.066117 [NOTICE] fpm_children_bury(), line 194: child 11725 stopped for tracing
Aug 29 11:46:59.066140 [NOTICE] fpm_php_trace(), line 139: about to trace 11725
Aug 29 11:46:59.066707 [NOTICE] fpm_php_trace(), line 167: finished trace of 11725

我通过执行下面的命令来运行django 项目 manage.py run_gunicorn --config=gunicorn.conf.py gunicorn默认是运行foreground的,但之前项目是没用supervisord,所以配置里daemon = True,supervisord是设计来运行foreground进程然后daemonize 该进程 监测exite status,所以gunicorn需要foreground运行 #daemon = True 参见:

场景:将一串包含若干<option>的html代码插入<select>中,在IE6-9下无法正确被添加,IE10表现如其他现代浏览器一样。

function fill_select4() {


    var sOpts = "";
    for (var i = 0; i < 100; i++) {
        sOpts += '<option value="' + i + '">' + i + '</option>';
    }

    select4.innerHTML = sOpts ;
} 

问题:在IE8下审查元素会发现select会像下例

<select name="select4" id="select4">
文字 - 0
</option/>
<option value="1"/>
文字 - 1
</option/>
...
</option/>

原因:未知

解决办法:

function fill_select1() {


    for(var i=0; i < 100; i++) {
        select1.options[i] = new Option('text'+i,'value'+i);
    }
}


function fill_select2() {


    var sOpts = "<select>";
    for (var i = 0; i < 100; i++) {
        sOpts += '<option value="' + i + '">' + i + '</option>';
    }

    select2.outerHTML = sOpts  + "</select>";
}


function fill_select3() {


    for(var i=0; i < 100; i++) {
       var oOption = document.createElement("OPTION");
       oOption.text="Option:  " + i;
       oOption.value=i;
       select3.add(oOption)
    }
}

场景:将一串html代码插入文档中,代码中包含一个表单,在IE8下该表单无法正确提交。

<form method="post" action="/url/">  
    <fieldset>
       <!--Form fields-->
    </fieldset>
</form>  

问题:假设需要插入文档中的表单像上例action的属性值以/结尾,IE8下会生成像下例的表单。

<form method="post" action="/url/"></form>  
<fieldset>  
   <!--Form fields-->
</fieldset>  

原因:若使用jQuery的html(),实际调用的是浏览器内建的innerHTML=<your_html_source>,而IE8在处理时会移除属性值两端的引号,变成action=/url />使表单闭合。

解决办法:html()传人jQuery对象,例如使用$(<your_html_source>),$(<source_container>).chidren().clone()…等方法产生jQuery对象。
相关文献:
jQuery: IE8 bug when adding a Form

根据Providing initial data for models,为原本想用数据库导出的sql,在app目录下建立sql目录,然后增加modelname.sql的文件,根据文档所说会在syncdb期间create table之后自动执行这个目录下的sql,但不知道是sql语句不符合要求还是怎样,总之没成功,即使我又删除了表用migrate命令也是。 反正这个特性django 2.0以后也不会支持,所以还是尝试采用正确的姿势吧。于是 根据下面两份资料结合Providing initial data for models我用了几个步骤搞定了。 Initial data and install-time code Prefer data migrations to initial data 1.

./manage.py dumpdata yourappname > init.json

在app文件夹下创建fixtures,将init.json放进来。 2.

./manage.py datamigration myapp mymigrationname

app的migrations文件夹下会多出xxxx_mymigrationname.py的文件,打开编辑。

1
2
3
def forwards(self, orm):
from django.core.management import call_command
call_command('loaddata', 'init.json')

3. 执行./manage.py migrate xlinks,然后就会执行新的mirgration Initial data and install-time code这篇文章说可以用initial_data命名的fixture file,在syncdb的时候执行,由于我已经创建好数据结构了,所以只好再增加个datamigration来执行migrate,那种方法也就没尝试。

嘛 首先安装 Xcode,20G下了一整晚,早知道买来就该下好的。

_imagingft.c:73:10: fatal error: ‘freetype/fterrors.h’ file not found

#include <freetype/fterrors.h>
^
1 error generated.
error: command ‘cc’ failed with exit status 1

http://stackoverflow.com/questions/20325473/error-installing-python-image-library-using-pip-on-mac-os-x-10-9

ln -s /usr/local/include/freetype2 /usr/local/include/freetype

clang: error: unknown argument: ‘-mno-fused-madd’ [-Wunused-command-line-argument-hard-error-in-future]

clang: note: this will be a hard error (cannot be downgraded to a warning) in the future

http://blog.csdn.net/xanxus46/article/details/22325253 环境:mac os x 10.9.2 python版本:2.7.5 经网上查证:http://www.tuicool.com/articles/zI7Vzu,貌似是mac os的Xcode从5.1起给编译器规定对于未知参数传入视为error,我们需要使用ARCHFLAGS将该error降级为warning,因此最后的安装命令应该如下:

sudo ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future easy_install PIL

这是我早期用的方法,用到了Timer,TimerTask去延迟,直到我看到http://stackoverflow.com/questions/8430805/android-clicking-twice-the-back-button-to-exit-activity 好吧,我的方法的确是蠢了点儿,先放在这儿吧。 声明activity成员变量及添加方法。

private byte back_count = 0;

Timer tExit = new Timer();
TimerTask tt;


private void beginTask() {
    if (tt != null) {
        tt.cancel();
    }
    tt = new TimerTask() {
        @Override
        public void run() {
            back_count = 0;
        }
    };
    tExit.schedule(tt, 2000);
}

重载onKeyDown

@Override
public boolean onKeyDown(int keyCode, KeyEvent event) {
if (keyCode == KeyEvent.KEYCODE_BACK) {
back_count++;
switch (back_count) {
case 1:
Toast.makeText(this, getString(R.string.confirmexit),
Toast.LENGTH_SHORT).show();
beginTask();

            break;
        case 2:
            tt.cancel();
            finish();
            overridePendingTransition(0, android.R.anim.fade_out);

            break;
        }
        if (back_count != 2)
            return false;
    }
    return super.onKeyDown(keyCode, event);
}
Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

×