GAE Upload large file to blobstore from an URL
我想将30MB到2GB的大型视频文件上传到具有外部URL的blob存储中，以访问此文件。
到目前为止，我尝试过类似的操作，但对于大文件却不起作用，我认为由于32MB的限制：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | //InputStream is = new URL(getUrlThumbnailFullSize(wistiaResponse)).openStream(); HttpURLConnection con = (HttpURLConnection) new URL("URL of the file here").openConnection(); con.setConnectTimeout(0); con.connect(); InputStream is = con.getInputStream(); //create file service to upload the video to the blob store FileService fileService = FileServiceFactory.getFileService(); //create the App Engine File; assign MIME type and name AppEngineFile file = fileService.createNewBlobFile("content type here","file name here"); //lock when uploading file boolean lock = true; //open channel FileWriteChannel writeChannel = fileService.openWriteChannel(file, lock); //upload file writeChannel.write(ByteBuffer.wrap(IOUtils.toByteArray(is))); writeChannel.closeFinally(); //release the lock lock = false; //get blob key BlobKey blobKey = FileServiceFactory.getFileService().getBlobKey(file); |
但是我有以下堆栈跟踪：
SEVERE: com.google.appengine.api.urlfetch.ResponseTooLargeException:
The response from url
http://embed.wistia.com/deliveries/a0f8dc8ed210b53152af601eb0af40d76cd2929f.bin
was too large. at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:57)
at
com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.fetchResponse(URLFetchServiceStreamHandler.java:417)
at
com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.getInputStream(URLFetchServiceStreamHandler.java:296)
at controller.CtrlWistia.download(CtrlWistia.java:149) at
controller.CtrlWistia.ajax(CtrlWistia.java:89) at
controller.CtrlAjax.main(CtrlAjax.java:66) at
vidaao.AjaxServlet.doPost(AjaxServlet.java:37) at
javax.servlet.http.HttpServlet.service(HttpServlet.java:637) at
javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
at
org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
at
org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
at
org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
at
org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.appengine.api.socket.dev.DevSocketFilter.doFilter(DevSocketFilter.java:74)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.appengine.tools.development.ResponseRewriterFilter.doFilter(ResponseRewriterFilter.java:123)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.appengine.tools.development.HeaderVerificationFilter.doFilter(HeaderVerificationFilter.java:34)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:61)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:125)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.appengine.tools.development.BackendServersFilter.doFilter(BackendServersFilter.java:97)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at
com.google.appengine.tools.development.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:94)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at
com.google.appengine.tools.development.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:409)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
请使用blobstoreService.createUploadUrl上传大文件。
[已更新]
啊，我想我明白了。
由于URLFetch服务的响应大小限制为32MB，因此目前无法实现。解决方法是，您可以从客户端javascript中的URL下载文件，然后使用blobstore服务创建的上传URL进行上传吗？
[已更新]
好的，您需要在服务器端实现此目标，这个问题对我来说还不明确。因此，请忘记createUploadUrl。
如果服务器支持Range标头，则可以多次请求较小的数据块，并将它们与所需的元数据(文件名，mimetype，blob键列表等)一起存储为单独的blob存储对象。
此外，如果要将它们合并到一个文件中，则可以使用Google Cloud Storage。使用可恢复上传选项，您可以多次将数据块发送到Google Cloud Storage。