hadoop security No groups available
Jul142015
【摘要】Hadoop NameNode 日志疯狂滚动，每天80G以上，日志中全是类似WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user xxx的报错，说明一下，任务是通过hive提交的。
google了大半天，没有建设性的意见，最后发现了这个链接中有这样一段话：
http://hakunamapdata.com/a-user-having-surprising-troubles-running-more-resource-intensive-hive-queries/
Possible fixes
How this problem could be solved? Obviously, a quick and dirty solution is to create an account on NameNode server for each user who accessing HDFS (directly, or by submitting MapReduce jobs to the cluster). However, for many reasons, you do not want to give everybody an account on NameNode server.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
2015-07-14 10:44:23,989 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user sudops 2015-07-14 10:44:23,991 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user sudops org.apache.hadoop.util.Shell$ExitCodeException: id: sudops：无此用户 at org.apache.hadoop.util.Shell.runCommand(Shell.java:255) at org.apache.hadoop.util.Shell.run(Shell.java:182) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375) at org.apache.hadoop.util.Shell.execCommand(Shell.java:461) at org.apache.hadoop.util.Shell.execCommand(Shell.java:444) at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:68) at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:45) at org.apache.hadoop.security.Groups.getGroups(Groups.java:79) at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1054) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.<init>(FSPermissionChecker.java:50) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5464) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkTraverse(FSNamesystem.java:5447) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:2168) at org.apache.hadoop.hdfs.server.namenode.NameNode.getFileInfo(NameNode.java:888) at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387) |
本文固定链接: https://www.sudops.com/hadoop-security-no-groups-available.html | 运维·速度
hadoop security No groups available：等您坐沙发呢！
发表评论
欢迎关注 运维·速度 微信公众号
